The gate for mysql-innodb-cluster is failing with this error:
======================================================================
FAIL: test_01_exporter_http_check (zaza.openstack.charm_tests.mysql.test_prometheus_mysql_exporter.PrometheusMySQLExporterTest)
Check exporter endpoint is working.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/zaza/openstack/charm_tests/mysql/test_prometheus_mysql_exporter.py", line 95, in test_01_exporter_http_check
self._check_exporter_http_check(unit)
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/tenacity/__init__.py", line 289, in wrapped_f
return self(f, *args, **kw)
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/tenacity/__init__.py", line 379, in __call__
do = self.iter(retry_state=retry_state)
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/tenacity/__init__.py", line 325, in iter
raise retry_exc.reraise()
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/tenacity/__init__.py", line 158, in reraise
raise self.last_attempt.result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/tenacity/__init__.py", line 382, in __call__
result = fn(*args, **kwargs)
File "/home/ubuntu/src/review.opendev.org/openstack/charm-mysql-innodb-cluster/src/.tox/func-target/lib/python3.8/site-packages/zaza/openstack/charm_tests/mysql/test_prometheus_mysql_exporter.py", line 81, in _check_exporter_http_check
self.fail(
AssertionError: Exporter permission not correct on 172.16.0.95
Ref: https://openstack-ci-reports.ubuntu.com/artifacts/549/888789/2/check/jammy/5499575/
Reproduced the issue in a lab environment:
$ curl -s http:// 10.5.0. 123:9104/ metrics | grep mysql_up mysqld- exporter/ common/ etc/my. cnf 9RZz3PyCpNZPGHc bP4PJM9jHqBXn6J SYXfrSsn3HzBk '9RZz3PyCpNZPGH cbP4PJM9jHqBXn6 JSYXfrSsn3HzBk' mysql.conf. d/mysqld. cnf' not found (OS errno 13 - Permission denied) @'10.5. 0.123' (using password: YES) '9RZz3PyCpNZPGH cbP4PJM9jHqBXn6 JSYXfrSsn3HzBk' @'10.5. 0.123' (using password: YES)
# HELP mysql_up Whether the MySQL server is up.
# TYPE mysql_up gauge
mysql_up 0
$ cat /var/snap/
[client]
user=prom_exporter
password=
port=3306
host=10.5.0.123
$ mysql -h 10.5.0.123 -u prom_exporter --password=
mysql: File '/etc/mysql/
mysql: [ERROR] Stopped processing the 'includedir' directive in file /etc/mysql/my.cnf at line 21.
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'prom_exporter'
$ sudo mysql -h 10.5.0.123 -u prom_exporter --password=
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'prom_exporter'
https:/ /pastebin. ubuntu. com/p/PPnWDVbxQ d/
Setting the password to the value stored in the leader databag fixes the issue:
mysql> SET PASSWORD FOR prom_exporter@ 10.5.0. 123 = '9RZz3PyCpNZPGH cbP4PJM9jHqBXn6 JSYXfrSsn3HzBk' ;
Query OK, 0 rows affected (0.01 sec)
$ curl -s http:// 10.5.0. 123:9104/ metrics | grep mysql_up
# HELP mysql_up Whether the MySQL server is up.
# TYPE mysql_up gauge
mysql_up 1
so the issue seems to be in the creation of the prom_exporter user.