Comment 20 for bug 1341656

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

{

    "build_id": "2014-09-15_00-01-46",
    "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
    "build_number": "8",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "b8d8189cc37d6d1b26f4479be6be7313beefb1c8",
    "production": "docker",
    "fuelmain_sha": "d7ed7973034bde73d3f42c000984423b59b2312b",
    "astute_sha": "f5fbd89d1e0e1f22ef9ab2af26da5ffbfbf24b13",
    "feature_groups": [
        "experimental"
    ],
    "release": "5.1",
    "release_versions": {
        "2014.1.1-5.1": {
            "VERSION": {
                "build_id": "2014-09-15_00-01-46",
                "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346",
                "build_number": "8",
                "api": "1.0",
                "nailgun_sha": "b8d8189cc37d6d1b26f4479be6be7313beefb1c8",
                "production": "docker",
                "fuelmain_sha": "d7ed7973034bde73d3f42c000984423b59b2312b",
                "astute_sha": "f5fbd89d1e0e1f22ef9ab2af26da5ffbfbf24b13",
                "feature_groups": [
                    "experimental"
                ],
                "release": "5.1",
                "fuellib_sha": "395fd9d20a003603cc9ad26e16cb13c1c45e24e6"
            }
        }
    },
    "fuellib_sha": "395fd9d20a003603cc9ad26e16cb13c1c45e24e6"

}

We have problems with oslo.messaging in our tests after restart nodes
Steps:
1. Create next cluster - Ubuntu, Simple, Flat nova-network, Cinder for volumes, Ceph for images, 1 controller, 1 compute, 2 cinder+ceph nodes
2. Deploy cluster
3. Reboot all nodes one by one
4. Open Health Check tab
5. Run tests

Expected - tests passed
Actual - tests on volume and instance creation failed. Lots of errors on compute node in nova logs:
2014-09-15 13:08:11 ERROR

oslo.messaging._drivers.impl_rabbit [req-c4c51ee4-f9b7-4e9e-a949-e694f3b21757 ] AMQP server on 10.108.52.4:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds.

And on cinder node in cinder logs
2014-09-15 07:40:19 ERROR

oslo.messaging._drivers.impl_rabbit [req-2fd99d10-2de0-478e-ab3e-cabc52935455 - - - - -] AMQP server on 10.108.52.4:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.

Logs are attached