ERR: (/Stage[main]/Nova::Api/Exec[nova-db-sync]) Failed to call refresh: /usr/bin/nova-manage db sync returned 1 instead of one of [0]

[Anastasia Palkina]

"build_id": "2015-02-07_22-55-01",
"ostf_sha": "6c046b69d29021524906109f18092363505ee222",
"build_number": "105",
"release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-02-07_22-55-01", "ostf_sha": "6c046b69d29021524906109f18092363505ee222", "build_number": "105", "api": "1.0", "nailgun_sha": "6d1769b21819f8fb4195f1bd9c44c038721ae3d4", "production": "docker", "python-fuelclient_sha": "521c2491f7f04f31d8c85db68499cd193d4904e3", "astute_sha": "7e6e6f9188bd69c603853b10d4a55149363323cc", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "", "fuellib_sha": "769af7fe30225cd15638ea2e6dffaa286bc06da1"}}},
"auth_required": true,
"api": "1.0",
"nailgun_sha": "6d1769b21819f8fb4195f1bd9c44c038721ae3d4",
"production": "docker",
"python-fuelclient_sha": "521c2491f7f04f31d8c85db68499cd193d4904e3",
"astute_sha": "7e6e6f9188bd69c603853b10d4a55149363323cc",
"feature_groups": ["mirantis"],
"release": "6.1",
"fuelmain_sha": "",
"fuellib_sha": "769af7fe30225cd15638ea2e6dffaa286bc06da1"

1. Create new environment (CentOS)
2. Choose nova-network, vlan manager
3. Choose Ceilometer
4. Add 1 controller, 1 compute, 1 cinder+mongo
5. Start deployment. It has failed
6. There is error on compute (node-11) in puppet.log

2015-02-09 15:54:54 ERR

 (/Stage[main]/Nova::Api/Exec[nova-db-sync]) Failed to call refresh: /usr/bin/nova-manage db sync returned 1 instead of one of [0]

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaSDBLMzNkQ0tWY00/view?usp=sharing

[Anastasia Palkina (apalkina)]

Reproduced this bug for Ubuntu on ISO #109

"build_id": "2015-02-09_22-54-44", "ostf_sha": "6c046b69d29021524906109f18092363505ee222", "build_number": "109", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-02-09_22-54-44", "ostf_sha": "6c046b69d29021524906109f18092363505ee222", "build_number": "109", "api": "1.0", "nailgun_sha": "71cb1dc573103e0faa6f54be687dd9fba57f6c8c", "production": "docker", "python-fuelclient_sha": "521c2491f7f04f31d8c85db68499cd193d4904e3", "astute_sha": "7e6e6f9188bd69c603853b10d4a55149363323cc", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "", "fuellib_sha": "4d04e7bb0472225e7d9f1f782f0525c8dba8cf06"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "71cb1dc573103e0faa6f54be687dd9fba57f6c8c", "production": "docker", "python-fuelclient_sha": "521c2491f7f04f31d8c85db68499cd193d4904e3", "astute_sha": "7e6e6f9188bd69c603853b10d4a55149363323cc", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "", "fuellib_sha": "4d04e7bb0472225e7d9f1f782f0525c8dba8cf06"

1. Create new environment (Ubuntu)
2. Choose nova-network, vlan
3. Choose Ceilometer
4. Add 1 controller, 1 compute, 1 cinder+mongo, 2 mongo
5. Start deployment. It has failed on compute (node-4) with the same error

[Anastasia Palkina (apalkina)]

Also there is error on controller for Ubuntu (node-3):

2015-02-10 11:25:40 ERR

 (/Stage[main]/Ceilometer::Db/Exec[ceilometer-dbsync]) Failed to call refresh: ceilometer-dbsync --config-file=/etc/ceilometer/ceilometer.conf returned 1 instead of one of [0]

[Ryan Moe (rmoe)]

The original compute problem seems to be this bug (1 controller + mongo will break deployment of other roles): https://bugs.launchpad.net/fuel/+bug/1411660. The ceilometer-dbsync problem looks like a different issue though.

[Bogdan Dobrelya (bogdando)]

According to the logs from #2, compute node issued nova db sync. Only controller nodes should make this step, IIRC.

[Bogdan Dobrelya (bogdando)]

ceilometer-dbsync could fail as there is no mongo MODULAR step reported in puppet-apply.log for node-3

[Bogdan Dobrelya (bogdando)]

My wrong, mongo MODULAR step was ok 2015-02-10T11:00:04.418433 node-6 ./node-6.domain.tld/puppet-apply.log:2015-02-10T11:00:04.418433+00:00 notice: (Scope(Class[main])) MODULAR: mongo.pp