[stein] Prometheus OpenStack Exporter Broken

Bug #1844851 reported by Joseph M
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Committed
Medium
Keith Plant
Stein
Fix Committed
Medium
Keith Plant
Train
Fix Committed
Medium
Keith Plant

Bug Description

**Bug Report**

What happened:
Two issues:
1. The clouds.yml template provided to prometheus-openstack-exporter is rendered with the wrong auth_url for multi region environments, using kolla_internal_fqdn instead of keystone_internal_url as described by the Kolla-Ansible multi region guide.

https://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/roles/prometheus/templates/clouds.yml.j2

2. The openstack-exporter version shipped in kolla stable/stein is 0.0.7 which does not retrieve stats for a standard stein deployment:

time="2019-09-21T01:26:21-04:00" level=info msg="Fetching list of services" source="nova.go:53"
time="2019-09-21T01:26:21-04:00" level=info msg="Fetching domains information" source="keystone.go:45"
time="2019-09-21T01:26:21-04:00" level=info msg="Fetching volumes info" source="cinder.go:54"
time="2019-09-21T01:26:21-04:00" level=info msg="Fetching images list" source="glance.go:39"
time="2019-09-21T01:26:21-04:00" level=info msg="Fetching floating ips list" source="neutron.go:47"
time="2019-09-21T01:26:21-04:00" level=error msg="Get /volumes: Get /volumes: unsupported protocol sch eme \"\"" source="cinder.go:57"
time="2019-09-21T01:26:21-04:00" level=info msg="Fetching services state information" source="cinder.g o:60"
time="2019-09-21T01:26:21-04:00" level=error msg="Get /os-services: Get /os-services: unsupported prot ocol scheme \"\"" source="cinder.go:63"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x82827d]

goroutine 81 [running]:
main.(*CinderExporter).Collect(0xc00001cac0, 0xc0002863c0)
        /app/cinder.go:66 +0x1dd
github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
        /app/vendor/github.com/prometheus/client_golang/prometheus/registry.go:434 +0x193
created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather
        /app/vendor/github.com/prometheus/client_golang/prometheus/registry.go:526 +0xe23

Setting "prometheus_openstack_exporter_tag: master" fixes the above issue due to the newer exporter version.

What you expected to happen:
It renders the clouds.yml template with the correct auth_url value and pulls stats correctly.

How to reproduce it (minimal and precise):
Set up a multi-region environment with the multi-region guide and set "enable_prometheus: yes"
https://docs.openstack.org/kolla-ansible/latest/user/multi-regions.html

**Environment**:
* OS (e.g. from /etc/os-release): CentOS Linux release 7.6.1810 (Core)
* Kernel (e.g. `uname -a`): 3.10.0-957.21.3.el7.x86_64
* Docker version if applicable (e.g. `docker version`): 19.03.2
* Kolla-Ansible version (e.g. `git head or tag or stable branch` or pip package version if using release): 8.0.2.dev4
* Docker image Install type (source/binary): source
* Docker image distribution: centos
* Are you using official images from Docker Hub or self built? Official
* If self built - Kolla version and environment used to build: NA
* Share your inventory file, globals.yml and other configuration files if relevant

Joseph M (noxoid)
summary: - Prometheus OpenStack Exporter Auth URL Incorrect
+ [stein] Prometheus OpenStack Exporter Broken
Revision history for this message
Mark Goddard (mgoddard) wrote :

1. Makes sense, please propose a fix.

2. I have seen this raised a few times. There has been a suggestion of bumping the openstack exporter to 0.0.9 on stable/stein. This fixes a common issue, and I think is in line with our stable branch policy since it's not a major version bump.

Changed in kolla-ansible:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Mark Goddard (mgoddard) wrote :

Here is a bug for the cinder issue: https://bugs.launchpad.net/kolla-ansible/+bug/1838391

Revision history for this message
Marcin Juszkiewicz (hrw) wrote :

https://review.opendev.org/#/c/689437/ bumped OpenStack Exporter to 0.6.0 version (in train). https://review.opendev.org/#/c/689659/ does same for stein.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.