OSTF test "RabbitMQ availability" has failed because method list-nodes includes only online nodes

[Anastasia Palkina]

1. Create new environment
2. Choose Neutron, VLAN
3. Choose Ceph for images
4. Add 3 controller, 1 compute, 1 cinder, 3 ceph
5. Move Storage network to eth1
6. Move Management network to eth2 and untag it
7. Deploy the environment. It was successful
8. Run OSTF tests. It was successful
9. SSH to promary controller and destroy system:
echo 1 > /proc/sys/kernel/sysrq
echo u > /proc/sysrq-trigger
10. Primary controller became 'offline' on UI but I can ssh to it
11. Start OSTF tests. Test "RabbitMQ availability" has failed with error: Number of RabbitMQ nodes is not equal to number of cluster nodes.

Cause is method list-nodes includes only online nodes (list_https://github.com/stackforge/fuel-ostf/blob/master/fuel_health/tests/ha/test_rabbit.py#L84)

May be we need to correct OSTF test.

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaUm1MSVpRSC1iSXM/view?usp=sharing

Controllers: node-12,11,18

"build_id": "298", "build_number": "298", "release_versions": {"2015.1.0-7.0": {"VERSION": {"build_id": "298", "build_number": "298", "api": "1.0", "fuel-library_sha": "0623b4daad438ceeb5dc41b10cdd3011795fff7e", "nailgun_sha": "d590b26dbb09785b8a8b3651b0ef69746fcf9991", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "486bde57cda1badb68f915f66c61b544108606f3", "astute_sha": "6c5b73f93e24cc781c809db9159927655ced5012", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "0623b4daad438ceeb5dc41b10cdd3011795fff7e", "nailgun_sha": "d590b26dbb09785b8a8b3651b0ef69746fcf9991", "feature_groups": ["mirantis"], "fuel-nailgun-agent_sha": "d7027952870a35db8dc52f185bb1158cdd3d1ebd", "openstack_version": "2015.1.0-7.0", "fuel-agent_sha": "082a47bf014002e515001be05f99040437281a2d", "production": "docker", "python-fuelclient_sha": "486bde57cda1badb68f915f66c61b544108606f3", "astute_sha": "6c5b73f93e24cc781c809db9159927655ced5012", "fuel-ostf_sha": "1f08e6e71021179b9881a824d9c999957fcc7045", "release": "7.0", "fuelmain_sha": "6b83d6a6a75bf7bca3177fcf63b2eebbf1ad0a85"

[Tatyanka (tatyana-leontovich)]

we do not assign high issues to the 7.0 - so move it to 8.0

[Tatyanka (tatyana-leontovich)]

this is expected behavior in ostf according to for those time when we run ostf there are 3 rabbitmq nodes running. So you can investigate why rabbit cluster was healthy in you case and report issue