Comment 14 for bug 1570805

During last swarm there were false positives of detection of this bug https://mirantis.testrail.com/index.php?/plans/view/12130
That is due to misleading error message, which should be made more verbose by https://review.openstack.org/#/c/322867/

Seems, related fix for object storage has resolved this issue https://bugs.launchpad.net/fuel/newton/+bug/1582646

Actual errors are:
https://mirantis.testrail.com/index.php?/tests/view/6475064
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.ha_neutron_tun_scale/124/testReport/(root)/neutron_tun_scalability/neutron_tun_scalability/
2016-05-30 03:41:47 DEBUG (test_haproxy) Dead backends ['horizon-ssl node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'keystone-1 node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'keystone-2 node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-metadata-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'cinder-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-glare node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'neutron node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-registry node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api-cfn node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api-cloudwatch node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-novncproxy node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ']

https://mirantis.testrail.com/index.php?/tests/view/6475080
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.controller_replacement/123/testReport/(root)/deploy_ha_neutron_tun_ctrl_replacement/deploy_ha_neutron_tun_ctrl_replacement/
2016-05-30 13:02:19 DEBUG (test_haproxy) Dead backends ['mysqld node-5 Status: DOWN/L4CON Sessions: 0 Rate: 0 ']
2016-05-30 13:02:19 ERROR (nose_storage_plugin) fuel_health.tests.ha.test_haproxy.HAProxyCheck.test_001_check_state_of_backends
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
    yield
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 601, in run
    testMethod()
  File "/usr/lib/python2.7/site-packages/fuel_health/tests/ha/test_haproxy.py", line 92, in test_001_check_state_of_backends
    "Step 2 failed: Some haproxy backend has down state.")
  File "/usr/lib/python2.7/site-packages/fuel_health/common/test_mixins.py", line 164, in verify_response_true
    self.fail(message.format(failed_step_msg, msg))
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
    raise self.failureException(msg)
AssertionError: Step 2 failed: Some haproxy backend has down state.. Please refer to OpenStack logs for more details.

https://mirantis.testrail.com/index.php?/tests/view/6475081
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.controller_replacement/123/testReport/(root)/deploy_ha_neutron_vlan_ctrl_replacement/
2016-05-30 05:44:18 DEBUG (test_haproxy) Dead backends ['horizon-ssl node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'keystone-1 node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'keystone-2 node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-metadata-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'cinder-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-glare node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'neutron node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'glance-registry node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api-cfn node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'heat-api-cloudwatch node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ', 'nova-novncproxy node-1 Status: DOWN/L4CON Sessions: 0 Rate: 0 ']
2016-05-30 05:44:18 ERROR (nose_storage_plugin) fuel_health.tests.ha.test_haproxy.HAProxyCheck.test_001_check_state_of_backends
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 67, in testPartExecutor
    yield
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 601, in run
    testMethod()
  File "/usr/lib/python2.7/site-packages/fuel_health/tests/ha/test_haproxy.py", line 92, in test_001_check_state_of_backends
    "Step 2 failed: Some haproxy backend has down state.")
  File "/usr/lib/python2.7/site-packages/fuel_health/common/test_mixins.py", line 164, in verify_response_true
    self.fail(message.format(failed_step_msg, msg))
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
    raise self.failureException(msg)
AssertionError: Step 2 failed: Some haproxy backend has down state.. Please refer to OpenStack logs for more details.

https://mirantis.testrail.com/index.php?/tests/view/6475138
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.plugins.fuel_plugin_example/126/testReport/(root)/deploy_neutron_example_ha/deploy_neutron_example_ha/
Install script fails with next message Traceback (most recent call last):
  File "/usr/bin/fuel", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/fuelclient/cli/error.py", line 121, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/fuelclient/cli/parser.py", line 247, in main
    parser.parse()
  File "/usr/lib/python2.7/site-packages/fuelclient/cli/parser.py", line 150, in parse
    actions[parsed_params.action].action_func(parsed_params)
  File "/usr/lib/python2.7/site-packages/fuelclient/cli/actions/base.py", line 56, in action_func
    method(params)
  File "/usr/lib/python2.7/site-packages/fuelclient/cli/actions/plugins.py", line 100, in install
    results = Plugins.install(file_path, force=params.force)
  File "/usr/lib/python2.7/site-packages/fuelclient/objects/plugins.py", line 340, in install
    name = plugin.name_from_file(plugin_path)
  File "/usr/lib/python2.7/site-packages/fuelclient/objects/plugins.py", line 166, in name_from_file
    return cls._get_metadata(file_path)['name']
  File "/usr/lib/python2.7/site-packages/fuelclient/objects/plugins.py", line 184, in _get_metadata
    plugin_tar = tarfile.open(plugin_path, 'r')
  File "/usr/lib64/python2.7/tarfile.py", line 1665, in open
    raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully

https://mirantis.testrail.com/index.php?/tests/view/6475069
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.bonding_ha/123/testReport/(root)/deploy_bonding_neutron_vlan/deploy_bonding_neutron_vlan/
Failed 7 OSTF tests; should fail 0 tests. Names of failed tests:
  - Check state of haproxy backends on controllers (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - Check data replication over mysql (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - Check if amount of tables in databases is the same on each node (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - Check galera environment state (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - Check pacemaker status (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - RabbitMQ availability (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct
  - RabbitMQ replication (failure) Can not set proxy for Health Check.Make sure that network configuration for controllers is correct