after removing 1st controller rabbitmq cluster take huge time for rebuild himself (about 180sec.)
[root@node-3 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@node-3' ...
[{nodes,[{disc,['rabbit@node-1','rabbit@node-2','rabbit@node-3']}]},
{running_nodes,['rabbit@node-2','rabbit@node-3']},
{partitions,[]}]
...done.
Horizon temporary feel sick
nova-compute go out permanently.
If we restart nova-conductor on ALL alive controllers _AND_ nova-compute -- nova-compute returns from darkness
If we have non-syncronized clock, we can see this fail:
after removing 1st controller rabbitmq cluster take huge time for rebuild himself (about 180sec.) [{disc, ['rabbit@ node-1' ,'rabbit@ node-2' ,'rabbit@ node-3' ]}]}, nodes,[ 'rabbit@ node-2' ,'rabbit@ node-3' ]},
[root@node-3 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@node-3' ...
[{nodes,
{running_
{partitions,[]}]
...done.
Horizon temporary feel sick
nova-compute go out permanently.
If we restart nova-conductor on ALL alive controllers _AND_ nova-compute -- nova-compute returns from darkness
If we have non-syncronized clock, we can see this fail:
nova-scheduler node-2.domain.tld internal enabled :-) 2014-04-30 17:46:51
nova-compute node-4.domain.tld nova enabled XXX 2014-04-30 17:44:13
nova-cert node-3.domain.tld internal enabled :-) 2014-04-30 17:46:51
In really it's a not fail, because timestamp of nova-compute was changed, and interpreted by nova-manage as fail because time not synched.