OSTF test fail with 400 bad request : availability zone is not available in test deploy_cinder_net_tmpl

[Tatyanka]

By default for all request we use default availability zone, but looks like it is not actual anymore

Deploy HA environment with Cinder, Neutron and network template

Scenario:

            1. Revert snapshot with 3 slaves
            2. Create cluster (HA) with Neutron VLAN/VXLAN/GRE
            3. Add 1 controller + cinder nodes
            4. Add 2 compute + cinder nodes
            5. Upload 'cinder' network template'
            6. Create custom network groups basing
               on template endpoints assignments
            7. Run network verification
            8. Deploy cluster
            9. Run network verification
            10. Run health checks (OSTF)
            11. Check L3 network configuration on slaves
            12. Check that services are listening on their networks only

Actual
on step 10 ostf tests are failed with:
...
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 433, in request
    raise exceptions.from_response(resp, body, url, method)
BadRequest: The requested availability zone is not available (HTTP 400) (Request-ID: req-875983ff-3b50-4b25-b086-4aa05571ce4b)

But at the same time there is no compute nodes in nova-manage service list:
root@node-1:~# nova-manage service list
No handlers could be found for logger "oslo_config.cfg"
Binary Host Zone Status State Updated_At
nova-cert node-1.test.domain.local internal enabled :-) 2015-11-27 15:18:32
nova-scheduler node-1.test.domain.local internal enabled :-) 2015-11-27 15:18:31
nova-conductor node-1.test.domain.local internal enabled :-) 2015-11-27 15:18:31
nova-consoleauth node-1.test.domain.local internal enabled :-) 2015-11-27 15:18:31

And availaility zones looks like:
root@node-1:~# nova availability-zone-list

+-----------------------------+----------------------------------------+
| Name | Status |
+-----------------------------+----------------------------------------+
| internal | available |
| |- node-1.test.domain.local | |
| | |- nova-conductor | enabled :-) 2015-11-27T15:12:31.000000 |
| | |- nova-consoleauth | enabled :-) 2015-11-27T15:12:31.000000 |
| | |- nova-scheduler | enabled :-) 2015-11-27T15:12:31.000000 |
| | |- nova-cert | enabled :-) 2015-11-27T15:12:32.000000 |
+-----------------------------+----------------------------------------+

In compute logs a lot a traceback about unavailability to rabbit,

so rabbit is unacceptable here by telnet too http://paste.openstack.org/show/480236/

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  openstack_version: "2015.1.0-8.0"
  api: "1.0"
  build_number: "212"
  build_id: "212"
  fuel-nailgun_sha: "13d1800be2452bd297555eedf051bda469b5663b"
  python-fuelclient_sha: "4f9a873b1ae68271bb848a1145b3cf912a176f4b"
  fuel-agent_sha: "f0e09fa7a30b5ab87e901601bfb6c1e339a6bdaa"
  fuel-nailgun-agent_sha: "b56f832abc18aee9a8c603fd6cc2055c5f4287bc"
  astute_sha: "d2c1b401816c6f0341902272f37018b9cec3c775"
  fuel-library_sha: "c892d2919620947225687ccde63d0e038784f0a2"
  fuel-ostf_sha: "5cf8c484a9a4f45f353468438cf98f1b5a5c5dfb"
  fuel-createmirror_sha: "9b335c8d551c87d788166947cb7ed519757881e8"
  fuelmenu_sha: "9627849843e84b7f01c44bd79898a8d62d96ce66"
  shotgun_sha: "34a081508b603723e14f9415a62373be39f3685f"
  network-checker_sha: "2c62cd52655ea6456ff6294fd63f18d6ea54fe38"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "daed38102b6074393821d675e3d7d866fcdb3da5"

[Aleksandr Didenko (adidenko)]

It heppens because of :
"No valid host was found. There are not enough hosts available."

There are no hypervisors in the openstack env. It's because computes are not able to connect to rabbitmq:

<183>Nov 27 16:04:20 node-3 nova-compute: Received recoverable error from kombu:
<179>Nov 27 16:04:20 node-3 nova-compute: AMQP server on 10.200.240.1:5673 is unreachable: timed out. Trying again in 6 seconds.

Messaging traffic has been moved to br-messaging:
10.200.240.0/24 dev br-messaging proto kernel scope link src 10.200.240.1

But on controller side connection to rabbitmq are blocked by firewall, only management network is allowed (note zero counters):

    0 0 ACCEPT tcp -- * * 10.109.6.0/24 0.0.0.0/0 multiport ports 4369,5672,5673,41055 /* 106 rabbitmq from 10.109.6.0/24 */

But not mgmt/messaging net role.

[Matthew Mosesohn (raytrac3r)]

We determine which networks are accessible on the management network via $management_nets = get_routable_networks_for_network_role($network_scheme, 'management')

When did we change network for management? How can we get the proper network IDs for this new messaging bridge?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/250908

[Aleksandr Didenko (adidenko)]

Considering network templates/roles, we need to use specific networks for specific firewall rules. It's better to do it directly in the modular task manifest than try to pass a bunch of additional parameters to openstak::firewall class. I've updated Sergey's patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/250908
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=8bb4bb7d77411cb9d78d40b4c4e6d96208dbb247
Submitter: Jenkins
Branch: master

commit 8bb4bb7d77411cb9d78d40b4c4e6d96208dbb247
Author: Sergey Vasilenko <email address hidden>
Date: Fri Nov 27 20:09:23 2015 +0300

    Move rules from openstack::firewall to firewall.pp

    Use openstack::firewall::multi_net and get_routable_networks_for_network_role()
    to collect firewall rules for multi-rack deployment.
    Also openstack::firewall usage replaced by osnailefacter granule directly.

    Change-Id: I87f5a6b64f8bc50514c1eca8ef0d70a5fe87d852
    Closes-bug: #1520613

[Tatyanka (tatyana-leontovich)]

verified 283 iso