[10.0] Can't deploy cluster: some nodes are in the Error state

[Ilya Bumarskov]

Fuel 10 build #1540

Steps to reproduce:
Deployment with 3 controllers, NeutronVlan, with add, delete,
           add/delete cinder+ceph node

        Scenario:
            1. Deploy cluster 3 controllers, 1 computes, 2 ceph + cinder,
               Neutron VLAN, cinder for volumes, ceph for images and ephemeral

Observed behaviour:

Cluster is not deployed: some nodes are in the Error state

CI jobs:
https://product-ci.infra.mirantis.net/job/10.0.acceptance.ubuntu.ha_scale_group_6/9/testReport/(root)/add_delete_compute_cinder_ceph_ephemeral/add_delete_compute_cinder_ceph_ephemeral/
https://product-ci.infra.mirantis.net/job/10.0.acceptance.ubuntu.ha_scale_group_3/9/testReport/(root)/add_delete_compute/add_delete_compute/

It seems something with rabbit on nodes:
(puppet-error.log)
Error: unable to connect to node 'rabbit@messaging-node-3': nodedown

DIAGNOSTICS
===========

attempted to contact: ['rabbit@messaging-node-3']

rabbit@messaging-node-3:
  * connected to epmd (port 4369) on messaging-node-3
  * epmd reports: node 'rabbit' not running at all
                  other nodes on messaging-node-3: ['rabbitmq-cli-86']
  * suggestion: start the node

current node details:
- node name: 'rabbitmq-cli-86@node-3'
- home dir: /var/lib/rabbitmq
- cookie hash: SK0DZu08GbWFxVW6dhCliw==

Error: unable to connect to node 'rabbit@messaging-node-3': nodedown

[Oleksiy Molchanov (omolchanov)]

Please reopen when reproduced.