Server manager provisioning stuck

[wenqing liang]

On R2.20-22, server manager provision is stuck. Reimaging went fine and contrail-status look ok.

root@cmbu-auto-esx1-lnx02:/etc# server-manager status server
{
    "server": [
        {
            "id": "cmbu-ceph-perf1",
            "ip_address": "10.87.140.197",
            "mac_address": "00:25:90:AB:9C:88",
            "status": "storage_started"
        },
        {
            "id": "cmbu-ceph-perf2",
            "ip_address": "10.87.140.198",
            "mac_address": "00:25:90:92:0D:54",
            "status": "storage-master_completed"
        },
        {
            "id": "cmbu-ceph-perf3",
            "ip_address": "10.87.140.199",
            "mac_address": "00:25:90:92:0E:6C",
            "status": "storage-master_completed"
        },
        {
            "id": "cmbu-ceph-perf4",
            "ip_address": "10.87.140.200",
            "mac_address": "00:25:90:92:0D:F2",
            "status": "provision_completed"
        }
    ]
}
root@cmbu-auto-esx1-lnx02:/etc#

[Dheeraj Gautam (dgautam)]

Failure with R2.20-23 Juno is happening for following reason:

root@cmbu-ceph-perf1:~# nova image-list
ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-934511e4-8fbb-4481-9458-ead7ffc377dd)

On running script manually, following logs are observed :

+ nova image-list
+ grep livemnfs
+ wc -l
ERROR (ClientException): An unexpected error prevented the server from fulfilling your request: (OperationalError) (2006, 'MySQL server has gone away') 'SELECT user.id AS user_id, user.name AS user_name, user.domain_id AS user_domain_id, user.password AS user_password, user.enabled AS user_enabled, user.extra AS user_extra, user.default_project_id AS user_default_project_id \nFROM user \nWHERE user.name = %s AND user.domain_id = %s' ('admin', 'default') (Disable debug mode to suppress these details.) (HTTP 500)
+ NUM_IMAGE_LIVEMNFS=0
+ glance image-create --name livemnfs --disk-format qcow2 --container-format ovf --file /tmp/livemnfs.qcow2 --is-public True
Error finding address for http://10.87.140.197:9292/v1/images: [Errno 111] Connection refused

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/10670
Submitter: Dheeraj Gautam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/10671
Submitter: Dheeraj Gautam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/10670
Committed: http://github.org/Juniper/contrail-puppet/commit/8c05c314e757bf2c32d7b396b0d1403370637477
Submitter: Zuul
Branch: R2.20

commit 8c05c314e757bf2c32d7b396b0d1403370637477
Author: root <email address hidden>
Date: Thu May 21 10:29:34 2015 -0700

SM-Storage: Added config to glance-api.conf
for known_stores for juno. This is required configuration for juno.

Change-Id: I9d17ee1fbff7eefc57696ddc8cf72954ce88046d
Partial-Bug: #1456049

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/10671
Committed: http://github.org/Juniper/contrail-puppet/commit/20881ef10a5f4c131b5269f0c4534867d982b9dd
Submitter: Zuul
Branch: master

commit 20881ef10a5f4c131b5269f0c4534867d982b9dd
Author: root <email address hidden>
Date: Thu May 21 10:29:34 2015 -0700

SM-Storage: Added config to glance-api.conf
for known_stores for juno. This is required configuration for juno.

Change-Id: I9d17ee1fbff7eefc57696ddc8cf72954ce88046d
Partial-Bug: #1456049

[Dheeraj Gautam (dgautam)]

There is a configuration change from ICE to Juno of glance-api. With check-ins that has been taken care.