featureset064-zed fails at overcloud deploy with different errors

[Ananya Banerjee]

featureset064-zed fails at overcloud deploy with different errors

- https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-zed/9abdf8c/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz
- https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-zed/670a585/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz
- https://logserver.rdoproject.org/42/46242/3/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-zed/b1fda3f/logs/undercloud/home/zuul/overcloud1_deploy.log.txt.gz

2022-11-28 05:43:25.489 27 DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643a884-27f9-437e-bfda-fc6a955f14c5 87df88ae624449999249d291c465640a - - all default -] RBAC: Authorization granted enforce_call /usr/lib/python3.9/site-packages/keystone/common/rbac_enforcer/enforcer.py:457
2022-11-28 05:43:25.500 27 ERROR keystone.server.flask.application [None req-5643a884-27f9-437e-bfda-fc6a955f14c5 87df88ae624449999249d291c465640a - - all default -] Could not find service: nova.: keystone.exception.ServiceNotFound: Could not find service: nova.

2022-11-26 16:30:37.653 23 DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e08571-1bb0-44bc-8ef6-05de1acaab49 f6ffdcf83df648f58d115b7744fe6c01 - - all default -] RBAC: Authorization granted enforce_call /usr/lib/python3.9/site-packages/keystone/common/rbac_enforcer/enforcer.py:457
2022-11-26 16:30:37.664 23 ERROR keystone.server.flask.application [None req-12e08571-1bb0-44bc-8ef6-05de1acaab49 f6ffdcf83df648f58d115b7744fe6c01 - - all default -] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service.

[Ananya Banerjee (frenzyfriday)]

Logs from keystone:

https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-zed/9abdf8c/logs/overcloud1-controller-0/var/log/containers/keystone/keystone.log.txt.gz

https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-zed/670a585/logs/overcloud1-controller-0/var/log/containers/keystone/keystone.log.txt.gz

2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application Traceback (most recent call last):
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/flask/app.py", line 1513, in full_dispatch_request
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application rv = self.dispatch_request()
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/flask/app.py", line 1499, in dispatch_request
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 467, in wrapper
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application resp = resource(*args, **kwargs)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/flask/views.py", line 83, in view
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application return self.dispatch_request(*args, **kwargs)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 582, in dispatch_request
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application resp = meth(*args, **kwargs)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/keystone/api/services.py", line 45, in get
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application return self._get_service(service_id)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/keystone/api/services.py", line 34, in _get_service
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id))
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/keystone/common/manager.py", line 115, in wrapped
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs)
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "<decorator-gen-17>", line 2, in get_service
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application File "/usr/lib/python3.9/site-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func
2022-11-28 05:43:26.034 23 ERROR keystone.server.flask.application ...

[Ananya Banerjee (frenzyfriday)]

2022-11-28 05:42:41.233 27 DEBUG keystone.common.rbac_enforcer.enforcer [None req-715255e6-4de5-4378-82ad-c9e3604f78bb 87df88ae624449999249d291c465640a - - all default -] RBAC: Authorizing `identity:get_project(project_id=admin)` enforce_call /usr/lib/python3.9/site-packages/keystone/common/rbac_enforcer/enforcer.py:449
2022-11-28 05:42:41.234 27 DEBUG keystone.common.rbac_enforcer.enforcer [None req-715255e6-4de5-4378-82ad-c9e3604f78bb 87df88ae624449999249d291c465640a - - all default -] RBAC: Authorization granted enforce_call /usr/lib/python3.9/site-packages/keystone/common/rbac_enforcer/enforcer.py:457
2022-11-28 05:42:41.245 27 ERROR keystone.server.flask.application [None req-715255e6-4de5-4378-82ad-c9e3604f78bb 87df88ae624449999249d291c465640a - - all default -] Could not find project: admin.: keystone.exception.ProjectNotFound: Could not find project: admin.

[Marios Andreou (marios-b)]

jobs are in better shape now.

we also have downstream equivalents running .

green upstream yesterday there https://review.rdoproject.org/zuul/build/1ebbbeb5bafe406b99475275eebe9269

moving invalid for now please move status back with a comment if you disagree