Comment 13 for bug 1861094

Revision history for this message
Jeffrey Chang (modern911) wrote :

SQA saw this in some of recent Yoga runs, posting some links and error below.
https://solutions.qa.canonical.com/testruns/94e7f7c0-6ac7-490b-a349-6048851aed10
https://solutions.qa.canonical.com/testruns/7c5c3299-895a-4da3-93ab-fc0945eee4c0
https://solutions.qa.canonical.com/testruns/c09547ef-0fb0-4a7e-bb85-62fe4c8e085d

We have some config issue about cpu-shared-set, cpu-dedicated-set. Not sure if they are related.
Will fix this see.

Errors from nova-compute.log

2023-07-13 05:11:19.100 34790 ERROR oslo.messaging._drivers.impl_rabbit [req-a12e35e8-e3a8-4b1d-bfa0-1709fd2f938c - - - - -] Connection failed: [Errno 111] ECONNREFUSED (retrying in 9.0 seconds): ConnectionRefusedError: [Errno 111] ECONNREFUSED

2023-07-13 06:13:28.436 1095570 WARNING nova.conductor.api [req-ccd0cb45-a009-45ba-ad2e-7bd8f9dd1e2a - - - - -] Timed out waiting for nova-conductor. Is it running? Or did this service start before nova-conductor? Reattempting establishment of nova-conductor connection...: oslo_m
essaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 578237a0fd7b466babfdf5fb382f49d9

2023-07-13 06:14:08.622 1133261 CRITICAL nova [req-31315d1a-f5d4-4fa9-80d1-96fbe385941f - - - - -] Unhandled error: keystoneauth1.exceptions.catalog.EndpointNotFound: ['internal', 'public'] endpoint for placement service in RegionOne region not found
2023-07-13 06:14:08.622 1133261 ERROR nova Traceback (most recent call last):
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/bin/nova-compute", line 10, in <module>
2023-07-13 06:14:08.622 1133261 ERROR nova sys.exit(main())
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/cmd/compute.py", line 59, in main
2023-07-13 06:14:08.622 1133261 ERROR nova server = service.Service.create(binary='nova-compute',
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/service.py", line 252, in create
2023-07-13 06:14:08.622 1133261 ERROR nova service_obj = cls(host, binary, topic, manager,
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/service.py", line 116, in __init__
2023-07-13 06:14:08.622 1133261 ERROR nova self.manager = manager_class(host=self.host, *args, **kwargs)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 631, in __init__
2023-07-13 06:14:08.622 1133261 ERROR nova self.reportclient = report.SchedulerReportClient()
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 234, in __init__
2023-07-13 06:14:08.622 1133261 ERROR nova self._client = self._create_client()
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/scheduler/client/report.py", line 277, in _create_client
2023-07-13 06:14:08.622 1133261 ERROR nova client = self._adapter or utils.get_sdk_adapter('placement')
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/nova/utils.py", line 985, in get_sdk_adapter
2023-07-13 06:14:08.622 1133261 ERROR nova return getattr(conn, service_type)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/openstack/service_description.py", line 87, in __get__
2023-07-13 06:14:08.622 1133261 ERROR nova proxy = self._make_proxy(instance)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/openstack/service_description.py", line 262, in _make_proxy
2023-07-13 06:14:08.622 1133261 ERROR nova found_version = temp_adapter.get_api_major_version()
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version
2023-07-13 06:14:08.622 1133261 ERROR nova return self.session.get_api_major_version(auth or self.auth, **kwargs)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1276, in get_api_major_version
2023-07-13 06:14:08.622 1133261 ERROR nova return auth.get_api_major_version(self, **kwargs)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version
2023-07-13 06:14:08.622 1133261 ERROR nova data = get_endpoint_data(discover_versions=discover_versions)
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 275, in get_endpoint_data
2023-07-13 06:14:08.622 1133261 ERROR nova endpoint_data = service_catalog.endpoint_data_for(
2023-07-13 06:14:08.622 1133261 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/access/service_catalog.py", line 462, in endpoint_data_for
2023-07-13 06:14:08.622 1133261 ERROR nova raise exceptions.EndpointNotFound(msg)
2023-07-13 06:14:08.622 1133261 ERROR nova keystoneauth1.exceptions.catalog.EndpointNotFound: ['internal', 'public'] endpoint for placement service in RegionOne region not found

2023-07-13 06:36:36.692 1474436 WARNING keystoneauth.identity.generic.base [req-7609443e-0609-4428-97f8-6f5b41fc00d4 - - - - -] Failed to discover available identity versions when contacting http://keystone-internal.silo4.lab1.solutionsqa:35357. Attempting to parse version from URL
.: keystoneauth1.exceptions.http.BadRequest: Bad Request (HTTP 400)
2023-07-13 06:36:36.696 1474436 CRITICAL nova [req-7609443e-0609-4428-97f8-6f5b41fc00d4 - - - - -] Unhandled error: keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is c
orrect. Bad Request (HTTP 400)
2023-07-13 06:36:36.696 1474436 ERROR nova Traceback (most recent call last):
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/generic/base.py", line 136, in _do_create_plugin
2023-07-13 06:36:36.696 1474436 ERROR nova disc = self.get_discovery(session,
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 608, in get_discovery
2023-07-13 06:36:36.696 1474436 ERROR nova return discover.get_discovery(session=session, url=url,
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/discover.py", line 1452, in get_discovery
2023-07-13 06:36:36.696 1474436 ERROR nova disc = Discover(session, url, authenticated=authenticated)
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/discover.py", line 535, in __init__
2023-07-13 06:36:36.696 1474436 ERROR nova self._data = get_version_data(session, url,
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/discover.py", line 102, in get_version_data
2023-07-13 06:36:36.696 1474436 ERROR nova resp = session.get(url, headers=headers, authenticated=authenticated)
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1141, in get
2023-07-13 06:36:36.696 1474436 ERROR nova return self.request(url, 'GET', **kwargs)
2023-07-13 06:36:36.696 1474436 ERROR nova File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 986, in request
2023-07-13 06:36:36.696 1474436 ERROR nova raise exceptions.from_response(resp, method, url)
2023-07-13 06:36:36.696 1474436 ERROR nova keystoneauth1.exceptions.http.BadRequest: Bad Request (HTTP 400)