ceilometer.agent [-] Continue after error from storage.objects.containers: Account HEAD failed: http://172.16.0.2:8080/... 503 Service Unavailable

[Anastasia Palkina]

"build_id": "2014-07-21_10-32-30",
"ostf_sha": "9863db951a6e159f4fa6e6861c8331e1af069cf8",
"build_number": "340",
"auth_required": false,
"api": "1.0",
"nailgun_sha": "bdd0bdec2b45eea843d559b7648bd5dca4873c66",
"production": "docker",
"fuelmain_sha": "539a5bf7493a5d14690a34bb18c3ad1c75b4f37f",
"astute_sha": "fd9b8e3b6f59b2727b1b037054f10e0dd7bd37f1",
"feature_groups": ["mirantis"],
"release": "5.1",
"fuellib_sha": "1ec799bc6c8b08b8c9c6243c426507cb7a46459b"

1. Create new environment (CentOS, HA mode)
2. Choose nova-network, flat
3. Choose Ceilometer
4. Add 3 controllers+mongo, compute, cinder
5. Start deployment. It was successful
6. Start OSTF tests. It was sucessful
7. But there are tracebacks in ceilometer-agent-central.log on first controller (node-1):

2014-07-21 12:49:23 WARNING

ceilometer.agent [-] Continue after error from storage.objects.containers: Account HEAD failed: http://172.16.0.2:8080/v1/AUTH_a84df6dfb83d4b5c91d1b646ead047e6 503 Service Unavailable
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent Traceback (most recent call last):
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent File "/usr/lib/python2.6/site-packages/ceilometer/agent.py", line 90, in poll_and_publish
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent resources=source_resources or agent_resources,
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent File "/usr/lib/python2.6/site-packages/ceilometer/objectstore/swift.py", line 130, in get_samples
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent for tenant, account in self._iter_accounts(manager.keystone, cache):
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent File "/usr/lib/python2.6/site-packages/ceilometer/objectstore/swift.py", line 61, in _iter_accounts
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent ksclient, cache))
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent File "/usr/lib/python2.6/site-packages/ceilometer/objectstore/swift.py", line 77, in _get_account_info
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent ksclient.auth_token))
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent File "/usr/lib/python2.6/site-packages/swiftclient/client.py", line 423, in head_account
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent http_response_content=body)
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent ClientException: Account HEAD failed: http://172.16.0.2:8080/v1/AUTH_a84df6dfb83d4b5c91d1b646ead047e6 503 Service Unavailable
2014-07-21 11:49:23.024 21915 TRACE ceilometer.agent

[Dina Belova (dbelova)]

Actually the reason is in the agent started before the swift (or writing logs in the case of no swift available) - really that makes no harm to the installation.

[Dina Belova (dbelova)]

@Ivan, please confirm this trace - if it's really so, this bug looks as a wish list for the ceilometer itself, not the Fuel

[Ivan Berezovskiy (iberezovskiy)]

Some of traces like in this bug can exist in ceilometer services logs because ceilometer agents can start before keystone, swift, nova and etc. It's not an issue - when deploy is ended all services work well. If this trace breakes deploy or blokes services work - we should fix it, but as I can see deploy and OSTF test was successful. So it don't need to fix

[Dina Belova (dbelova)]

Agree with Ivan - while the env is not fully woken up, it's normal to see the warnings like that.