ceilometer-upgrade error

[Yang Shi]

I'm trying to install and configure ceilometer on the controller node according to the install guide.
When I run the command:
# ceilometer-upgrade --skip-metering-database
2017-11-29 15:59:06.536 4097 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2017-11-29 15:59:06.536 4097 INFO ceilometer.cmd.storage [-] Skipping metering database upgrade
2017-11-29 15:59:07.165 4097 CRITICAL ceilometer [-] ConnectFailure: Unable to establish connection to http://controller:8041/v1/resource_type/ceph_account: HTTPConnectionPool(host='controller', port=8041): Max retries exceeded with url: /v1/resource_type/ceph_account (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f7bb28eec10>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-11-29 15:59:07.165 4097 ERROR ceilometer Traceback (most recent call last):
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/bin/ceilometer-upgrade", line 10, in <module>
2017-11-29 15:59:07.165 4097 ERROR ceilometer sys.exit(upgrade())
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/ceilometer/cmd/storage.py", line 53, in upgrade
2017-11-29 15:59:07.165 4097 ERROR ceilometer gnocchi_client.upgrade_resource_types(conf)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/ceilometer/gnocchi_client.py", line 113, in upgrade_resource_types
2017-11-29 15:59:07.165 4097 ERROR ceilometer gnocchi.resource_type.get(name=name)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/v1/resource_type.py", line 44, in get
2017-11-29 15:59:07.165 4097 ERROR ceilometer headers={'Content-Type': "application/json"}).json()
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/v1/base.py", line 37, in _get
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.client.api.get(*args, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 217, in get
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.request(url, 'GET', **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/gnocchiclient/client.py", line 35, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 142, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer return self.session.request(url, method, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line 101, in inner
2017-11-29 15:59:07.165 4097 ERROR ceilometer return wrapped(*args, **kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 616, in request
2017-11-29 15:59:07.165 4097 ERROR ceilometer resp = send(**kwargs)
2017-11-29 15:59:07.165 4097 ERROR ceilometer File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 690, in _send_request
2017-11-29 15:59:07.165 4097 ERROR ceilometer raise exceptions.ConnectFailure(msg)
2017-11-29 15:59:07.165 4097 ERROR ceilometer ConnectFailure: Unable to establish connection to http://controller:8041/v1/resource_type/ceph_account: HTTPConnectionPool(host='controller', port=8041): Max retries exceeded with url: /v1/resource_type/ceph_account (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f7bb28eec10>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-11-29 15:59:07.165 4097 ERROR ceilometer

[gordon chung (chungg)]

what does gnocchi-api logs say? you should probably paste your gnocchi.conf and ceilometer.conf files if you aren't able to find the error in gnocchi-api

[Frank (f-sardis)]

I have experienced this also. I followed the Pike guide for ceilometer and I get
Unable to establish connection to http://controller:8041/v1/resource_type/ceph
when i try to run ceilometer-upgrade --skip-metering-database
I do not use ceph in my openstack and i do not see anything in the ceilometer.conf that mentions ceph so it is not clear where this comes from.

I have tried gnocchi via apt and via pip. Same problem always. Gnocchi api doesn't seem to be installed via pip and via apt it does not register as a service at all so I cannot (re)start it.

Also the guide specifies using coordination_url = redis://controller:6379 but this produces a similar error during gnocchi-upgrade in that the redis service cannot be reached and the guide does not mention anything about installing it (nor does the gnocchi install guide). If i comment out the line, gnocchi-upgrade proceeds without a problem and uses the file driver.

There is no clarity in the guide on what is needed and for what reason which makes it difficult to troubleshoot.

[gordon chung (chungg)]

in the docs[1], there's a step that says: systemctl start openstack-gnocchi-api.service. can you run systemctl status openstack-gnocchi-api.service to check status? or journalctl to read logs?

if you deployed behind httpd, can you show the logs there?

regarding coordination_url, yes, that appears to be unexplained. i'll add explanation.

[1] https://docs.openstack.org/ceilometer/latest/install/install-base-rdo.html

[gordon chung (chungg)]

if you're wondering why it references ceph, it's becasuse ceilometer will create resource types (schemas) for all the possible resources it can theoretically monitor whether you enable them in ceilometer or not.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/540870

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/540870
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=bd60ada44ae1948ef668fdf145879b5b215ae2f4
Submitter: Zuul
Branch: master

commit bd60ada44ae1948ef668fdf145879b5b215ae2f4
Author: gord chung <email address hidden>
Date: Mon Feb 5 14:30:21 2018 +0000

    add note explaining gnocchi coordination_url

    Change-Id: I632663679d0649e1d5912aef98a65c610fdeaac2
    Partial-Bug: #1745806