glance-api.conf points to sqlite driver

[Michele Baldessari]

I suspect https://review.opendev.org/c/openstack/puppet-tripleo/+/786644 is the root cause here of glance not working correctly in master:

The failed
https://logserver.rdoproject.org/44/786644/11/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0025d8b/logs/overcloud-controller-0/var/lib/config-data/puppet-generated/glance_api/etc/glance/glance-api.conf.txt.gz has:
connection=sqlite:///var/lib/glance/glance.sqlite

whereas it should be:
connection=mysql+pymysql://glance:.....

In fact RDO was red exactly because of this I think:
https://logserver.rdoproject.org/44/786644/11/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0025d8b/logs/undercloud/var/log/tempest/tempestconf.log.txt.gz

2021-08-04 01:48:36.207 304572 INFO tempest.lib.common.rest_client [req-47aa3d5b-0586-4cd4-b791-e3071fe8c8f8 ] Request (main): 500 GET https://10.0.0.5:13292/v2/images 0.152s
2021-08-04 01:48:36.208 304572 CRITICAL tempest [-] Unhandled error: tempest.lib.exceptions.ServerFault: Got server fault
Details: The server has either erred or is incapable of performing the requested operation.<br /><br />
2021-08-04 01:48:36.208 304572 ERROR tempest Traceback (most recent call last):
2021-08-04 01:48:36.208 304572 ERROR tempest File "/bin/discover-tempest-config", line 10, in <module>
...
2021-08-04 01:48:36.208 304572 ERROR tempest

On the node we see:
https://logserver.rdoproject.org/44/786644/11/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/0025d8b/logs/overcloud-controller-2/var/log/containers/glance/api.log.txt.gz

2021-08-04 01:48:36.068 27 DEBUG glance.api.middleware.version_negotiation [-] Matched version: v2 process_request /usr/lib/python3.6/site-packages/glance/api/middleware/version_negotiation.py:69
2021-08-04 01:48:36.068 27 DEBUG glance.api.middleware.version_negotiation [-] new path /v2/images process_request /usr/lib/python3.6/site-packages/glance/api/middleware/version_negotiation.py:70
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi [req-47aa3d5b-0586-4cd4-b791-e3071fe8c8f8 08f3a5b415df4736a7e89ab88e9d97b0 28feb35bc5364e2287c527dca9ea24fc - default default] Caught error: (sqlite3.OperationalError) no such table: images
[SQL: SELECT anon_1.images_created_at AS anon_1_images_created_at, anon_1.images_updated_at AS anon_1_images_updated_at, anon_1.images_deleted_at AS anon_1_images_deleted_at, anon_1.images_deleted AS anon_1_images_deleted, anon_1.images_id AS anon_1_images_id, anon_1.images_name AS anon_1_images_name, anon_1.images_disk_format AS anon_1_images_disk_format, anon_1.images_container_format AS anon_1_images_container_format, anon_1.images_size AS anon_1_images_size, anon_1.images_virtual_size AS anon_1_images_virtual_size, anon_1.images_status AS anon_1_images_status, anon_1.images_visibility AS anon_1_images_visibility, anon_1.images_checksum AS anon_1_images_checksum, anon_1.images_os_hash_algo AS anon_1_images_os_hash_algo, anon_1.images_os_hash_value AS anon_1_images_os_hash_value, anon_1.images_min_disk AS anon_1_images_min_disk, anon_1.images_min_ram AS anon_1_images_min_ram, anon_1.images_owner AS anon_1_images_owner, anon_1.images_protected AS anon_1_images_protected, anon_1.images_os_hidden AS anon_1_images_os_hidden, image_properties_1.created_at AS image_properties_1_created_at, image_properties_1.updated_at AS image_properties_1_updated_at, image_properties_1.deleted_at AS image_properties_1_deleted_at, image_properties_1.deleted AS image_properties_1_deleted, image_properties_1.id AS image_properties_1_id, image_properties_1.image_id AS image_properties_1_image_id, image_properties_1.name AS image_properties_1_name, image_properties_1.value AS image_properties_1_value, image_tags_1.created_at AS image_tags_1_created_at, image_tags_1.updated_at AS image_tags_1_updated_at, image_tags_1.deleted_at AS image_tags_1_deleted_at, image_tags_1.deleted AS image_tags_1_deleted, image_tags_1.id AS image_tags_1_id, image_tags_1.image_id AS image_tags_1_image_id, image_tags_1.value AS image_tags_1_value, image_locations_1.created_at AS image_locations_1_created_at, image_locations_1.updated_at AS image_locations_1_updated_at, image_locations_1.deleted_at AS image_locations_1_deleted_at, image_locations_1.deleted AS image_locations_1_deleted, image_locations_1.id AS image_locations_1_id, image_locations_1.image_id AS image_locations_1_image_id, image_locations_1.value AS image_locations_1_value, image_locations_1.meta_data AS image_locations_1_meta_data, image_locations_1.status AS image_locations_1_status
FROM (SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.virtual_size AS images_virtual_size, images.status AS images_status, images.visibility AS images_visibility, images.checksum AS images_checksum, images.os_hash_algo AS images_os_hash_algo, images.os_hash_value AS images_os_hash_value, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected, images.os_hidden AS images_os_hidden
FROM images
WHERE images.os_hidden = 0 AND images.deleted = 0 AND images.status IN (?, ?, ?, ?, ?, ?, ?, ?) AND (images.owner = ? OR images.visibility != ?) ORDER BY images.created_at DESC, images.id DESC
 LIMIT ? OFFSET ?) AS anon_1 LEFT OUTER JOIN image_properties AS image_properties_1 ON anon_1.images_id = image_properties_1.image_id LEFT OUTER JOIN image_tags AS image_tags_1 ON anon_1.images_id = image_tags_1.image_id LEFT OUTER JOIN image_locations AS image_locations_1 ON anon_1.images_id = image_locations_1.image_id ORDER BY anon_1.images_created_at DESC, anon_1.images_id DESC]
[parameters: ('active', 'saving', 'queued', 'pending_delete', 'deleted', 'deactivated', 'importing', 'uploading', '28feb35bc5364e2287c527dca9ea24fc', 'community', 25, 0)]
(Background on this error at: http://sqlalche.me/e/13/e3q8): oslo_db.exception.DBNonExistentTable: (sqlite3.OperationalError) no such table: images
[SQL: SELECT anon_1.images_created_at AS anon_1_images_created_at, anon_1.images_updated_at AS anon_1_images_updated_at, anon_1.images_deleted_at AS anon_1_images_deleted_at, anon_1.images_deleted AS anon_1_images_deleted, anon_1.images_id AS anon_1_images_id, anon_1.images_name AS anon_1_images_name, anon_1.images_disk_format AS anon_1_images_disk_format, anon_1.images_container_format AS anon_1_images_container_format, anon_1.images_size AS anon_1_images_size, anon_1.images_virtual_size AS anon_1_images_virtual_size, anon_1.images_status AS anon_1_images_status, anon_1.images_visibility AS anon_1_images_visibility, anon_1.images_checksum AS anon_1_images_checksum, anon_1.images_os_hash_algo AS anon_1_images_os_hash_algo, anon_1.images_os_hash_value AS anon_1_images_os_hash_value, anon_1.images_min_disk AS anon_1_images_min_disk, anon_1.images_min_ram AS anon_1_images_min_ram, anon_1.images_owner AS anon_1_images_owner, anon_1.images_protected AS anon_1_images_protected, anon_1.images_os_hidden AS anon_1_images_os_hidden, image_properties_1.created_at AS image_properties_1_created_at, image_properties_1.updated_at AS image_properties_1_updated_at, image_properties_1.deleted_at AS image_properties_1_deleted_at, image_properties_1.deleted AS image_properties_1_deleted, image_properties_1.id AS image_properties_1_id, image_properties_1.image_id AS image_properties_1_image_id, image_properties_1.name AS image_properties_1_name, image_properties_1.value AS image_properties_1_value, image_tags_1.created_at AS image_tags_1_created_at, image_tags_1.updated_at AS image_tags_1_updated_at, image_tags_1.deleted_at AS image_tags_1_deleted_at, image_tags_1.deleted AS image_tags_1_deleted, image_tags_1.id AS image_tags_1_id, image_tags_1.image_id AS image_tags_1_image_id, image_tags_1.value AS image_tags_1_value, image_locations_1.created_at AS image_locations_1_created_at, image_locations_1.updated_at AS image_locations_1_updated_at, image_locations_1.deleted_at AS image_locations_1_deleted_at, image_locations_1.deleted AS image_locations_1_deleted, image_locations_1.id AS image_locations_1_id, image_locations_1.image_id AS image_locations_1_image_id, image_locations_1.value AS image_locations_1_value, image_locations_1.meta_data AS image_locations_1_meta_data, image_locations_1.status AS image_locations_1_status
FROM (SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.virtual_size AS images_virtual_size, images.status AS images_status, images.visibility AS images_visibility, images.checksum AS images_checksum, images.os_hash_algo AS images_os_hash_algo, images.os_hash_value AS images_os_hash_value, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected, images.os_hidden AS images_os_hidden
FROM images
WHERE images.os_hidden = 0 AND images.deleted = 0 AND images.status IN (?, ?, ?, ?, ?, ?, ?, ?) AND (images.owner = ? OR images.visibility != ?) ORDER BY images.created_at DESC, images.id DESC
 LIMIT ? OFFSET ?) AS anon_1 LEFT OUTER JOIN image_properties AS image_properties_1 ON anon_1.images_id = image_properties_1.image_id LEFT OUTER JOIN image_tags AS image_tags_1 ON anon_1.images_id = image_tags_1.image_id LEFT OUTER JOIN image_locations AS image_locations_1 ON anon_1.images_id = image_locations_1.image_id ORDER BY anon_1.images_created_at DESC, anon_1.images_id DESC]
[parameters: ('active', 'saving', 'queued', 'pending_delete', 'deleted', 'deactivated', 'importing', 'uploading', '28feb35bc5364e2287c527dca9ea24fc', 'community', 25, 0)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi Traceback (most recent call last):
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi cursor, statement, parameters, context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi cursor.execute(statement, parameters)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi sqlite3.OperationalError: no such table: images
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi The above exception was the direct cause of the following exception:
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi Traceback (most recent call last):
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/common/wsgi.py", line 1354, in __call__
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi request, **action_args)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/common/wsgi.py", line 1397, in dispatch
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return method(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/api/v2/images.py", line 522, in index
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi member_status=member_status)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/api/authorization.py", line 117, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi images = self.image_repo.list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/domain/proxy.py", line 89, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi items = self.base.list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/api/policy.py", line 185, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return super(ImageRepoProxy, self).list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/domain/proxy.py", line 89, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi items = self.base.list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/domain/proxy.py", line 89, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi items = self.base.list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/domain/proxy.py", line 89, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi items = self.base.list(*args, **kwargs)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/db/__init__.py", line 88, in list
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi member_status=member_status, return_tag=True)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib/python3.6/site-packages/glance/db/sqlalchemy/api.py", line 712, in image_get_all
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi for image in query.all():
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3373, in all
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return list(self)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return self._execute_and_instances(context)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi result = conn.execute(querycontext.statement, self._params)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return meth(self, multiparams, params)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi return connection._execute_clauseelement(self, multiparams, params)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi distilled_params,
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi e, statement, parameters, cursor, context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1508, in _handle_dbapi_exception
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi raise exception
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi cursor, statement, parameters, context
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi cursor.execute(statement, parameters)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi oslo_db.exception.DBNonExistentTable: (sqlite3.OperationalError) no such table: images
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi [SQL: SELECT anon_1.images_created_at AS anon_1_images_created_at, anon_1.images_updated_at AS anon_1_images_updated_at, anon_1.images_deleted_at AS anon_1_images_deleted_at, anon_1.images_deleted AS anon_1_images_deleted, anon_1.images_id AS anon_1_images_id, anon_1.images_name AS anon_1_images_name, anon_1.images_disk_format AS anon_1_images_disk_format, anon_1.images_container_format AS anon_1_images_container_format, anon_1.images_size AS anon_1_images_size, anon_1.images_virtual_size AS anon_1_images_virtual_size, anon_1.images_status AS anon_1_images_status, anon_1.images_visibility AS anon_1_images_visibility, anon_1.images_checksum AS anon_1_images_checksum, anon_1.images_os_hash_algo AS anon_1_images_os_hash_algo, anon_1.images_os_hash_value AS anon_1_images_os_hash_value, anon_1.images_min_disk AS anon_1_images_min_disk, anon_1.images_min_ram AS anon_1_images_min_ram, anon_1.images_owner AS anon_1_images_owner, anon_1.images_protected AS anon_1_images_protected, anon_1.images_os_hidden AS anon_1_images_os_hidden, image_properties_1.created_at AS image_properties_1_created_at, image_properties_1.updated_at AS image_properties_1_updated_at, image_properties_1.deleted_at AS image_properties_1_deleted_at, image_properties_1.deleted AS image_properties_1_deleted, image_properties_1.id AS image_properties_1_id, image_properties_1.image_id AS image_properties_1_image_id, image_properties_1.name AS image_properties_1_name, image_properties_1.value AS image_properties_1_value, image_tags_1.created_at AS image_tags_1_created_at, image_tags_1.updated_at AS image_tags_1_updated_at, image_tags_1.deleted_at AS image_tags_1_deleted_at, image_tags_1.deleted AS image_tags_1_deleted, image_tags_1.id AS image_tags_1_id, image_tags_1.image_id AS image_tags_1_image_id, image_tags_1.value AS image_tags_1_value, image_locations_1.created_at AS image_locations_1_created_at, image_locations_1.updated_at AS image_locations_1_updated_at, image_locations_1.deleted_at AS image_locations_1_deleted_at, image_locations_1.deleted AS image_locations_1_deleted, image_locations_1.id AS image_locations_1_id, image_locations_1.image_id AS image_locations_1_image_id, image_locations_1.value AS image_locations_1_value, image_locations_1.meta_data AS image_locations_1_meta_data, image_locations_1.status AS image_locations_1_status
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi FROM (SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.virtual_size AS images_virtual_size, images.status AS images_status, images.visibility AS images_visibility, images.checksum AS images_checksum, images.os_hash_algo AS images_os_hash_algo, images.os_hash_value AS images_os_hash_value, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected, images.os_hidden AS images_os_hidden
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi FROM images
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi WHERE images.os_hidden = 0 AND images.deleted = 0 AND images.status IN (?, ?, ?, ?, ?, ?, ?, ?) AND (images.owner = ? OR images.visibility != ?) ORDER BY images.created_at DESC, images.id DESC
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi LIMIT ? OFFSET ?) AS anon_1 LEFT OUTER JOIN image_properties AS image_properties_1 ON anon_1.images_id = image_properties_1.image_id LEFT OUTER JOIN image_tags AS image_tags_1 ON anon_1.images_id = image_tags_1.image_id LEFT OUTER JOIN image_locations AS image_locations_1 ON anon_1.images_id = image_locations_1.image_id ORDER BY anon_1.images_created_at DESC, anon_1.images_id DESC]
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi [parameters: ('active', 'saving', 'queued', 'pending_delete', 'deleted', 'deactivated', 'importing', 'uploading', '28feb35bc5364e2287c527dca9ea24fc', 'community', 25, 0)]
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi (Background on this error at: http://sqlalche.me/e/13/e3q8)
2021-08-04 01:48:36.183 27 ERROR glance.common.wsgi
2021-08-04 01:48:36.205 27 INFO eventlet.wsgi.server [req-47aa3d5b-0586-4cd4-b791-e3071fe8c8f8 08f3a5b415df4736a7e89ab88e9d97b0 28feb35bc5364e2287c527dca9ea24fc - default default] 172.17.0.160 - - [04/Aug/2021 01:48:36] "GET /v2/images HTTP/1.1" 500 399 0.138389

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803526

[Takashi Kajinami (kajinamit)]

I think the problem here is https://review.opendev.org/c/openstack/puppet-tripleo/+/786644 included glance::api::db BEFORE glance::api, and tht still use deprecated parameters in glance::api.
Because pick doesn't work directly with hiera and ignores any parameters from glance::api class unless it is already included.

I have submitted a patch to fix usage of the deprecated parameters from glance::api. Let's see how it goes.
https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803526

[yatin (yatinkarel)]

Affecting promotion pipeline too https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/ace37fd/logs/undercloud/var/log/tempest/tempestconf.log.txt.gz

[Marios Andreou (marios-b)]

@ykarel yes we already have a bug for this https://bugs.launchpad.net/tripleo/+bug/1938558 and also a cix card didn't need to add promotion blocker here

[EDIT]: not so sure they are duplicate any more so leaving both for now and unmarking as duplicate

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803526
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/169e4e98347c00c30f9827e0e3ac3e1ccaa6d155
Submitter: "Zuul (22348)"
Branch: master

commit 169e4e98347c00c30f9827e0e3ac3e1ccaa6d155
Author: Takashi Kajinami <email address hidden>
Date: Thu Aug 5 08:12:10 2021 +0900

    Replace deprecated glance::api::database_connection

    Database parameters in glance::api were deprecated in favor of
    duplicated parameters in glance::api::db[1].

    [1] 10f14f29b10d3cb427e74ef2ad2c45be1ae97842

    Closes-Bug: #1938932
    Change-Id: I8a0515e2d4e99fef7f4b3fa8fe0990f3f061cb52

[Marios Andreou (marios-b)]

for the record, was also hitting this on fs35 there [1]

2021-08-04 11:27:50.779 402042 INFO tempest.lib.common.rest_client [req-74cc8081-6700-465c-bd4c-71d2f11a9727 ] Request (main): 500 GET https://[2001:db8:fd00:1000::5]:13292/v2/images 1.118s
2021-08-04 11:27:50.780 402042 CRITICAL tempest [-] Unhandled error: tempest.lib.exceptions.ServerFault: Got server fault

and the wrong config in glance api conf [2]

connection=sqlite:///var/lib/glance/glance.sqlite

The fix is now merged so trying a rerun of this fs35 job with [3] to confirm (and hopefully promote that hash ;))

[1] https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/527d998/logs/undercloud/var/log/tempest/tempestconf.log.txt.gz
[2] https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/527d998/logs/overcloud-controller-0/var/lib/config-data/puppet-generated/glance_api/etc/glance/glance-api.conf.txt.gz
[3] https://review.rdoproject.org/r/c/testproject/+/34860

[Marios Andreou (marios-b)]

The fix at [1] merged and it seems to have fixed the job - the test run at [2] was successful.

Question - do we need this for wallaby? Since [3] merged last October I would expect we need it for wallaby and victoria at least.

However, why are we not hitting this on featureset 1 wallaby and why does it already have the correct config without needing [1]? For example at [4] we have:

connection=mysql+pymysql://glance:vOQdcogFd1l82nZ0QjuRq6ivI@172.17.0.44/glance?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo

And the job is green. However in the wallaby branch [5] we still have the deprecated:

           - glance::api::database_connection:

[1] https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803526
[2] https://review.rdoproject.org/r/c/testproject/+/34860/1#message-d1ad522c7ad41255e09e120fb787ef4d84eb54b9
[3] https://review.opendev.org/c/openstack/puppet-glance/+/758910
[4] https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-wallaby/841d5b9/logs/overcloud-controller-0/var/lib/config-data/puppet-generated/glance_api/etc/glance/glance-api.conf.txt.gz
[5] https://opendev.org/openstack/tripleo-heat-templates/src/commit/e76573eb072e9760fa86ed635dcc07c710615cad/deployment/glance/glance-api-container-puppet.yaml#L452

[Marios Andreou (marios-b)]

copy/paste from #tripleo just now, but yeah we need the wallaby cherrypick. the reason we aren't seeing the issue there yet is because the puppet-tripleo patch https://review.opendev.org/c/openstack/puppet-tripleo/+/786644 hasn't been merged on wallaby yet

12:20 < tkajinam> marios|ruck, the problem was caused by
                  https://review.opendev.org/c/openstack/puppet-tripleo/+/786644 which was merged recently, and
                  that fix is not yet backported to wallaby. that's why we don't see the issue in W
12:20 < tkajinam> We should have merge that fix in tht first then this puppet-tripleo patch
12:21 < marios|ruck> tkajinam: i see ok then it makes more sense ... but then why does wallaby get the 'right'
                     config with connection=mysql+pymysql
12:22 < marios|ruck> tkajinam: (probably related/because puppet-tripleo/+/786644 isn't on wallaby yet? )
12:22 < tkajinam> marios|ruck, yeah
12:22 < Tengu> probably better backporting, especially since wallaby will be osp-17...
12:22 < tkajinam> marios|ruck, puppet-glance automatically convert glance::api::database_connection to
                  glance::api::db::database_connection
12:23 < tkajinam> but this logic doesn't work if we include glance::api::db BEFORE glance::api
12:23 < tkajinam> that what I made in that puppet-tripleo patch :-P
12:23 < tkajinam> that's *
12:23 < marios|ruck> tkajinam: ah i see ok it makes sense now thanks
12:24 < marios|ruck> tkajinam: so first we need the cherrypick to tht then the cherrypick to pupppet-tripleo

[Marios Andreou (marios-b)]

master merged waiting for wallaby @ https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803676

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803676
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/1c9f86f6f002091afc7accff889913b873c46603
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 1c9f86f6f002091afc7accff889913b873c46603
Author: Takashi Kajinami <email address hidden>
Date: Thu Aug 5 08:12:10 2021 +0900

    Replace deprecated glance::api::database_connection

    Database parameters in glance::api were deprecated in favor of
    duplicated parameters in glance::api::db[1].

    [1] 10f14f29b10d3cb427e74ef2ad2c45be1ae97842

    Related-Bug: #1938932
    Change-Id: I8a0515e2d4e99fef7f4b3fa8fe0990f3f061cb52
    (cherry picked from commit 169e4e98347c00c30f9827e0e3ac3e1ccaa6d155)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.