CeilometerAlarmTest fails with RBAC Authorization Failed (HTTP 403)

[Rabi Mishra]

from the logs it looks like ceilometer nova client auth issue.

http://logs.openstack.org/14/234514/18/check/gate-heat-dsvm-functional-orig-mysql/b7c33b4/logs/screen-h-eng.txt.gz#_2015-12-07_03_27_36_111

http://logs.openstack.org/53/250053/2/gate/gate-heat-dsvm-functional-orig-mysql/d9e1c80/logs/screen-ceilometer-acentral.txt.gz#_2015-12-06_22_29_09_937

Logs:

'network.\\1.\\2.rate', 'unit': '\\1/s'}}}}], 'name': 'network_sink'}]}
2015-12-06 22:29:09.930 32229 INFO ceilometer.pipeline [req-f54134d3-1a6b-451b-b211-119756d11c32 admin - - - -] detected decoupled pipeline config format
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client [req-f54134d3-1a6b-451b-b211-119756d11c32 admin - - - -] HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa8ec985c50>: Failed to establish a new connection: [Errno 111] Connection refused',))
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client Traceback (most recent call last):
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/opt/stack/new/ceilometer/ceilometer/nova_client.py", line 47, in with_logging
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client return func(*args, **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/opt/stack/new/ceilometer/ceilometer/nova_client.py", line 164, in instance_get_all
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client search_opts=search_opts)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 609, in list
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client "servers")
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 72, in _list
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client _resp, body = self.api.client.get(url)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 446, in get
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client return self._cs_request(url, 'GET', **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 402, in _cs_request
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client self.authenticate()
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 555, in authenticate
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client auth_url = self._v2_auth(auth_url)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 646, in _v2_auth
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client return self._authenticate(url, body)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 659, in _authenticate
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 397, in _time_request
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client resp, body = self.request(url, method, **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 366, in request
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client response = session.request(method=method, url=url, **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 468, in request
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client resp = self.send(prep, **send_kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 576, in send
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client r = adapter.send(request, **kwargs)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 423, in send
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client raise ConnectionError(e, request=request)
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa8ec985c50>: Failed to establish a new connection: [Errno 111] Connection refused',))
2015-12-06 22:29:09.937 32229 ERROR ceilometer.nova_client

[Sergey Kraynev (skraynev)]

Temporary skip was merged https://review.openstack.org/#/c/254014/

[Thomas Herve (therve)]

I think http://logs.openstack.org/14/234514/18/check/gate-heat-dsvm-functional-orig-mysql/b7c33b4/logs/apache/aodh.txt.gz is the relevant error, not that agent central traceback.

[gordon chung (chungg)]

ceilometer doesn't have alarms. it's aodh. you should rename that. :)

[Sergey Kraynev (skraynev)]

@gordon: thx :) I forgot about this change :)

[gordon chung (chungg)]

resolved: https://review.openstack.org/#/c/254078/