[Update] Openstack stops working after Rollback cluster

[Andrey Sledzinskiy]

Reproduced on custom upgrade tarball from 09/10/2014
{

    "ostf_sha": "1de6ed1c0b72f6687ffb4bebc2c939b135a88e34",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "82091e0d61f252619a0842d0f8debb6b602a61fe",
    "production": "docker",
    "fuelmain_sha": "b07d8e86bba02266e02bb4e2b0629b07dab51351",
    "astute_sha": "cd89b4869910d98f7cdc33ea77086f90d9f1985c",
    "feature_groups": [
        "experimental"
    ],
    "release": "5.1",
    "release_versions": {
        "2014.1-5.0": {
            "VERSION": {
                "build_id": "2014-05-27_05-51-41",
                "mirantis": "yes",
                "build_number": "26",
                "ostf_sha": "a8b7660082a6f152794c610d6abe30d360fd577d",
                "nailgun_sha": "bd09f89ef56176f64ad5decd4128933c96cb20f4",
                "production": "docker",
                "api": "1.0",
                "fuelmain_sha": "505741e4f431f85a8d0252fc42754d10c0326c1a",
                "astute_sha": "a7eac46348dc77fc2723c6fcc3dbc66cc1a83152",
                "release": "5.0",
                "fuellib_sha": "2f79c0415159651fc1978d99bd791079d1ae4a06"
            }
        },
        "2014.1.1-5.0.2": {
            "VERSION": {
                "build_id": "2014-09-09_10-19-11",
                "ostf_sha": "2969c1ad443b632e815bb1f01149c3800cd7aa3a",
                "build_number": "72",
                "api": "1.0",
                "nailgun_sha": "c18a21381843dffe807b254a4ff96eec259953cb",
                "production": "docker",
                "fuelmain_sha": "b1e46c8c3ac86a8eafac225507294226322160ea",
                "astute_sha": "6db5f5031b74e67b92fcac1f7998eaa296d68025",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "5.0.2",
                "fuellib_sha": "b538a2baa1cfc3c345e6f7d52f5ca191abdd2b74"
            }
        },
        "2014.1.1-5.1": {
            "VERSION": {
                "ostf_sha": "1de6ed1c0b72f6687ffb4bebc2c939b135a88e34",
                "api": "1.0",
                "nailgun_sha": "82091e0d61f252619a0842d0f8debb6b602a61fe",
                "production": "docker",
                "fuelmain_sha": "b07d8e86bba02266e02bb4e2b0629b07dab51351",
                "astute_sha": "cd89b4869910d98f7cdc33ea77086f90d9f1985c",
                "feature_groups": [
                    "experimental"
                ],
                "release": "5.1",
                "fuellib_sha": "e3f947cc4142210499a282bc5f183c333552fa23"
            }
        }
    },
    "fuellib_sha": "e3f947cc4142210499a282bc5f183c333552fa23"

}

Steps:
1. Create next 5.0-26 cluster - CentOS, HA, Neutron Vlan, Cinder for volumes, 3 controlers, 2 compute, 1 cinder node
2. Deploy cluster
3. After successful deployment upgrade master to 5.1
4. Update exisiting cluster to 5.0.2
5. After rollback cluster back to 5.0
6. Run OSTF

Expected - OSTF passed
Actual - all Openstack related tests failed with
fuel_health.common.test_mixins: DEBUG: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/fuel_health/common/test_mixins.py", line 186, in verify
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/fuel_health/nmanager.py", line 760, in _list_networks
    networks = client.networks.list()
  File "/usr/lib/python2.6/site-packages/novaclient/v1_1/networks.py", line 51, in list
    return self._list("/os-networks", "networks")
  File "/usr/lib/python2.6/site-packages/novaclient/base.py", line 64, in _list
    _resp, body = self.api.client.get(url)
  File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 283, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 260, in _cs_request
    **kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 242, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/lib/python2.6/site-packages/novaclient/client.py", line 236, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-82274827-1ca7-42d4-9340-fdd8a53b4e50)

Logs are attached

[Andrey Sledzinskiy (asledzinskiy)]

Bug was also reproduced after the same steps on HA neutron GRE env
Logs are attached