Comment 0 for bug 1436812

Revision history for this message
Alexey Khivin (akhivin) wrote : Entire RabbitMQ cluster downtime when primary controller goes down

Multinode with HA with 3 controllers

Steps to reproduce

1) find primary controller (crm status)
2) On the other two controllers start

watch "rabbitmqctl eval 'rabbit_misc:which_applications().' |grep rabbit,"

to see whats happening with RabbitMQ cluster and rabbit application. You should see the line

{rabbit,"RabbitMQ","3.3.5"},

3) Turn off the primary controller and see whats happening with other two controllers

Because of currrent HA implementation you will see that rabbitmqctl could not connect to the node
something like that
Error: unable to connect to node 'rabbit@node-10': nodedown
or
line with rabbitmq application could disappear

The normal behaviour I believe should be
No disconnection on two others controllers and no full cluster failover when primary controller goes down

Also in the
https://bugs.launchpad.net/fuel/+bug/1435250
noted
3) destroy the master, node-3
Expected: new master election, failover with full downtime - no nodes can process AMQP connections

I think It should be improved

[root@fuel ~]# fuel --f
DEPRECATION WARNING: file /etc/fuel/client/config.yaml is found and will be used as a source for settings. However, it deprecated and will not be used by default in the ongoing version of python-fuelclient.
api: '1.0'
astute_sha: 4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9
auth_required: true
build_id: 2015-03-23_15-29-20
build_number: '218'
feature_groups:
- mirantis
fuellib_sha: a0265ae47bb2307a6967a3f1dd06fe222c561265
fuelmain_sha: a05ab877af31924585c81081f45305700961458e
nailgun_sha: 7c100f47450ea1a910e19fa09f78d586cb2bc0d3
ostf_sha: a4cf5f218c6aea98105b10c97a4aed8115c15867
production: docker
python-fuelclient_sha: 3624051242c83fdbdd1df9a0e466797c06b75043
release: '6.1'
release_versions:
  2014.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: 4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9
      build_id: 2015-03-23_15-29-20
      build_number: '218'
      feature_groups:
      - mirantis
      fuellib_sha: a0265ae47bb2307a6967a3f1dd06fe222c561265
      fuelmain_sha: a05ab877af31924585c81081f45305700961458e
      nailgun_sha: 7c100f47450ea1a910e19fa09f78d586cb2bc0d3
      ostf_sha: a4cf5f218c6aea98105b10c97a4aed8115c15867
      production: docker
      python-fuelclient_sha: 3624051242c83fdbdd1df9a0e466797c06b75043
      release: '6.1'