[image based] HA deployment failed by timeout with errors in puppet log(/Stage[main]/Cinder::Api/Exec[cinder-manage db_sync]) Failed to call refresh: Command exceeded timeout

[Andrey Sledzinskiy]

custom iso with image-based provisioning
{

    "build_id": "2014-10-07_13-04-02",
    "ostf_sha": "45fe4038ba861673f3a6877d64582c2f66607d30",
    "build_number": "465",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "6ba1397de286eddd9d404f93303e34320849ad90",
    "production": "docker",
    "fuelmain_sha": "d4aa7399ba28fcd643ae0b07b7ce0a71e87c6749",
    "astute_sha": "157fcf2037205f0696816cb41b9db6a04fdc1ab4",
    "feature_groups": [
        "mirantis",
        "experimental",
        "imagebased"
    ],
    "release": "6.0",
    "release_versions": {
        "2014.2-6.0": {
            "VERSION": {
                "build_id": "2014-10-07_13-04-02",
                "ostf_sha": "45fe4038ba861673f3a6877d64582c2f66607d30",
                "build_number": "465",
                "api": "1.0",
                "nailgun_sha": "6ba1397de286eddd9d404f93303e34320849ad90",
                "production": "docker",
                "fuelmain_sha": "d4aa7399ba28fcd643ae0b07b7ce0a71e87c6749",
                "astute_sha": "157fcf2037205f0696816cb41b9db6a04fdc1ab4",
                "feature_groups": [
                    "mirantis",
                    "experimental",
                    "imagebased"
                ],
                "release": "6.0",
                "fuellib_sha": "e43e9ba98f33f8fced49dcbac35847b42c04bc82"
            }
        }
    },
    "fuellib_sha": "e43e9ba98f33f8fced49dcbac35847b42c04bc82"

}

Steps:
1. Create next cluster - Centos with image based provisioning, HA, Flat Nova-network, 3 controllers, 2 compute, 1 cinder nodes
2. Start cluster deployment

Expected - cluster was successfully deployed
Actual - Depoyment failed by timeout on node-2 and node-3
Errors in puppet logs:
2014-10-08 07:37:06 ERR

 (/Stage[main]/Cinder::Api/Exec[cinder-manage db_sync]) /usr/lib/ruby/site_ruby/1.8/puppet/type/exec.rb:98:in `sync'

2014-10-08 07:37:06 ERR

 (/Stage[main]/Cinder::Api/Exec[cinder-manage db_sync]) /usr/lib/ruby/site_ruby/1.8/puppet/util/errors.rb:97:in `fail'

2014-10-08 07:37:06 ERR

 (/Stage[main]/Cinder::Api/Exec[cinder-manage db_sync]) Command exceeded timeout

2014-10-08 07:37:06 ERR

 (/Stage[main]/Cinder::Api/Exec[cinder-manage db_sync]) Failed to call refresh: Command exceeded timeout

Logs are attached

[Andrey Sledzinskiy (asledzinskiy)]

verified on {

    "build_id": "2014-11-17_17-53-34",
    "ostf_sha": "82465a94eed4eff1fc8d8e1f2fb7e9993c22f068",
    "build_number": "504",
    "auth_required": true,
    "api": "1.0",
    "nailgun_sha": "8d23d1b1bcd9213a70a40c38c3c1486d215d40b5",
    "production": "docker",
    "fuelmain_sha": "8d4943d5ead7a894d4af5e10172510fa60eeed84",
    "astute_sha": "65eb911c38afc0e23d187772f9a05f703c685896",
    "feature_groups": [
        "mirantis"
    ],
    "release": "6.0",
    "release_versions": {
        "2014.2-6.0": {
            "VERSION": {
                "build_id": "2014-11-17_17-53-34",
                "ostf_sha": "82465a94eed4eff1fc8d8e1f2fb7e9993c22f068",
                "build_number": "504",
                "api": "1.0",
                "nailgun_sha": "8d23d1b1bcd9213a70a40c38c3c1486d215d40b5",
                "production": "docker",
                "fuelmain_sha": "8d4943d5ead7a894d4af5e10172510fa60eeed84",
                "astute_sha": "65eb911c38afc0e23d187772f9a05f703c685896",
                "feature_groups": [
                    "mirantis"
                ],
                "release": "6.0",
                "fuellib_sha": "8a0ceff90777af75a3f9363a57185e608f3ee10d"
            }
        }
    },
    "fuellib_sha": "8a0ceff90777af75a3f9363a57185e608f3ee10d"

}