Comment 9 for bug 1394635

Revision history for this message
Kirill Omelchenko (kirill-omelchenko) wrote : Re: Handshake_timeout of rabbit after shutdown primary controller

I had a kind of this issue on a virtual env (5.1.1 - #45).

3x Controllers, 2x Computes, 2x CEPH-storage

- after successfull setup, shutdown the primary controller.

as a result we have errors output by crm status:
[root@node-2 ~]# crm status
Last updated: Mon Dec 1 10:14:55 2014
Last change: Mon Dec 1 10:14:18 2014 via crm_attribute on node-3.test.domain.local
Stack: classic openais (with plugin)
Current DC: node-3.test.domain.local - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
3 Nodes configured, 3 expected votes
17 Resources configured

Online: [ node-2.test.domain.local node-3.test.domain.local ]
OFFLINE: [ node-1.test.domain.local ]

 vip__management_old (ocf::mirantis:ns_IPaddr2): Started node-2.test.domain.local
 vip__public_old (ocf::mirantis:ns_IPaddr2): Started node-2.test.domain.local
 Clone Set: clone_ping_vip__public_old [ping_vip__public_old]
     Started: [ node-2.test.domain.local node-3.test.domain.local ]
 Clone Set: clone_p_mysql [p_mysql]
     Started: [ node-2.test.domain.local node-3.test.domain.local ]
 Master/Slave Set: master_p_rabbitmq-server [p_rabbitmq-server]
     Masters: [ node-3.test.domain.local ]
     Slaves: [ node-2.test.domain.local ]
 Clone Set: clone_p_haproxy [p_haproxy]
     Started: [ node-2.test.domain.local node-3.test.domain.local ]
 Clone Set: clone_p_openstack-heat-engine [p_openstack-heat-engine]
     Started: [ node-2.test.domain.local node-3.test.domain.local ]

Failed actions:
    ping_vip__public_old_monitor_20000 on node-2.test.domain.local 'unknown error' (1): call=64, status=Timed Out, last-rc-change='Fri Nov 28 16:45:15 2014', queued=0ms, exec=0ms
    p_mysql_monitor_120000 on node-2.test.domain.local 'unknown error' (1): call=90, status=complete, last-rc-change='Fri Nov 28 14:54:30 2014', queued=0ms, exec=0ms
    p_mysql_monitor_120000 on node-3.test.domain.local 'unknown error' (1): call=105, status=complete, last-rc-change='Fri Nov 28 14:53:22 2014', queued=0ms, exec=0ms

Impacts instance creation and all related tests/actions both via OSTF and manualy.
Diagnostic snapshot: https://copy.com/4KpLdOkhteZMHiOm