| 2022-06-04 07:35:20 |
odie |
bug |
|
|
added bug |
| 2022-06-04 07:38:13 |
odie |
summary |
octavia ERROR oslo_messaging.rpc.server |
octavia error: oslo_messaging.rpc.server |
|
| 2022-06-04 07:42:10 |
odie |
description |
Deployed openstack bundles and octavia using overlay and everything works except an error when creating Load balancer.
ERROR:
Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server
o-hm0 port doesn't have an IPv4
re deployed and set create-mgmt-network: false by default it creates IPv6 management network for Octavia and get an IP but the same error was encountered
Also tried the octavia rev 39 with same error. May I know which octavia versions are working?
========================================================================================================
Octavia bundles:
barbican - ver. 14.0.0 - rev. 67
octavia - ver - 10.0.0 rev. 71
octavia-ovn-chassis - ver. 22.03.0 - rev. 21
glance-simplestreams-sync - rev 68
octavia-diskimage-retrofit - ver 0.9.12 - rev. 45
==========================================================================================================
openstack maas-controller maas-one/default 2.9.31 unsupported 06:55:00Z
App Version Status Scale Charm Store Channel Rev OS Message
barbican 14.0.0 active 1 barbican charmhub yoga/stable 67 ubuntu Unit is ready
barbican-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
barbican-vault 3.10.0 active 1 barbican-vault charmhub stable 27 ubuntu Unit is ready
ceph-mon 15.2.16 active 3 ceph-mon charmhub quincy/stable 106 ubuntu Unit is ready and clu
ceph-osd 15.2.16 active 4 ceph-osd charmhub quincy/stable 534 ubuntu Unit is ready (1 OSD)
ceph-radosgw 17.2.0 active 1 ceph-radosgw charmhub quincy/stable 526 ubuntu Unit is ready
cinder 16.4.2 active 1 cinder charmhub yoga/stable 554 ubuntu Unit is ready
cinder-ceph 16.4.2 active 1 cinder-ceph charmhub yoga/stable 502 ubuntu Unit is ready
cinder-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
dashboard-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance 24.0.0 active 1 glance charmhub yoga/stable 544 ubuntu Unit is ready
glance-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance-simplestreams-sync active 1 glance-simplestreams-sync charmhub yoga/stable 68 ubuntu Unit is ready (Glance/22 06:25:28, metadata uploaded to object store)
keystone 21.0.0 active 1 keystone charmhub yoga/stable 568 ubuntu Application Ready
keystone-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
mysql-innodb-cluster 8.0.29 active 3 mysql-innodb-cluster charmhub 8.0/stable 24 ubuntu Unit is ready: Mode: nd can tolerate up to ONE failure.
neutron-api 16.4.2 active 1 neutron-api charmhub yoga/stable 526 ubuntu Unit is ready
neutron-api-plugin-ovn 16.4.2 active 1 neutron-api-plugin-ovn charmhub yoga/stable 29 ubuntu Unit is ready
neutron-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
nova-cloud-controller 21.2.4 active 1 nova-cloud-controller charmhub yoga/stable 601 ubuntu Unit is ready
nova-compute 21.2.4 active 3 nova-compute charmhub yoga/stable 588 ubuntu Unit is ready
nova-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
ntp 3.5 active 3 ntp charmhub stable 47 ubuntu chrony: Ready
octavia 10.0.0 active 1 octavia charmhub yoga/stable 71 ubuntu Unit is ready
octavia-dashboard 9.0.0 active 1 octavia-dashboard charmhub yoga/stable 44 ubuntu Unit is ready
octavia-diskimage-retrofit 0.9.12 active 1 octavia-diskimage-retrofit charmhub yoga/stable 45 ubuntu Unit is ready (Image 75e34ed11dc9 retrofitting completed at 06/04/22 03:51:24)
octavia-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
octavia-ovn-chassis 22.03.0 active 1 ovn-chassis charmhub stable 21 ubuntu Unit is ready
openstack-dashboard 22.1.0 active 1 openstack-dashboard charmhub yoga/stable 536 ubuntu Unit is ready
ovn-central 22.03.0 active 3 ovn-central charmhub 22.03/stable 31 ubuntu Unit is ready (northd
ovn-chassis 20.03.2 active 3 ovn-chassis charmhub 22.03/stable 46 ubuntu Unit is ready
placement 7.0.0 active 1 placement charmhub yoga/stable 49 ubuntu Unit is ready
placement-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
rabbitmq-server 3.8.2 active 1 rabbitmq-server charmhub 3.9/stable 149 ubuntu Unit is ready
vault 1.7.9 active 1 vault charmhub 1.7/stable 68 ubuntu Unit is ready (active
vault-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready |
Deployed openstack bundles and octavia using overlay and everything works except an error when creating Load balancer.
ERROR:
Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server
o-hm0 port doesn't have an IPv4
re deployed and set create-mgmt-network: false by default it creates IPv6 management network for Octavia and get an IP but the same error was encountered
Also tried the octavia rev 39 with same error. May I know which octavia versions are working?
========================================================================================================
Octavia bundles:
barbican - ver. 14.0.0 - rev. 67
octavia - ver - 10.0.0 rev. 71
octavia-ovn-chassis - ver. 22.03.0 - rev. 21
glance-simplestreams-sync - rev 68
octavia-diskimage-retrofit - ver 0.9.12 - rev. 45
==========================================================================================================
openstack maas-controller maas-one/default 2.9.31 unsupported 06:55:00Z
App Version Status Scale Charm Store Channel Rev OS Message
barbican 14.0.0 active 1 barbican charmhub yoga/stable 67 ubuntu Unit is ready
barbican-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
barbican-vault 3.10.0 active 1 barbican-vault charmhub stable 27 ubuntu Unit is ready
ceph-mon 15.2.16 active 3 ceph-mon charmhub quincy/stable 106 ubuntu Unit is ready and clu
ceph-osd 15.2.16 active 4 ceph-osd charmhub quincy/stable 534 ubuntu Unit is ready (1 OSD)
ceph-radosgw 17.2.0 active 1 ceph-radosgw charmhub quincy/stable 526 ubuntu Unit is ready
cinder 16.4.2 active 1 cinder charmhub yoga/stable 554 ubuntu Unit is ready
cinder-ceph 16.4.2 active 1 cinder-ceph charmhub yoga/stable 502 ubuntu Unit is ready
cinder-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
dashboard-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance 24.0.0 active 1 glance charmhub yoga/stable 544 ubuntu Unit is ready
glance-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance-simplestreams-sync active 1 glance-simplestreams-sync charmhub yoga/stable 68 ubuntu Unit is ready (Glance/22 06:25:28, metadata uploaded to object store)
keystone 21.0.0 active 1 keystone charmhub yoga/stable 568 ubuntu Application Ready
keystone-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
mysql-innodb-cluster 8.0.29 active 3 mysql-innodb-cluster charmhub 8.0/stable 24 ubuntu Unit is ready: Mode: nd can tolerate up to ONE failure.
neutron-api 16.4.2 active 1 neutron-api charmhub yoga/stable 526 ubuntu Unit is ready
neutron-api-plugin-ovn 16.4.2 active 1 neutron-api-plugin-ovn charmhub yoga/stable 29 ubuntu Unit is ready
neutron-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
nova-cloud-controller 21.2.4 active 1 nova-cloud-controller charmhub yoga/stable 601 ubuntu Unit is ready
nova-compute 21.2.4 active 3 nova-compute charmhub yoga/stable 588 ubuntu Unit is ready
nova-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
ntp 3.5 active 3 ntp charmhub stable 47 ubuntu chrony: Ready
octavia 10.0.0 active 1 octavia charmhub yoga/stable 71 ubuntu Unit is ready
octavia-dashboard 9.0.0 active 1 octavia-dashboard charmhub yoga/stable 44 ubuntu Unit is ready
octavia-diskimage-retrofit 0.9.12 active 1 octavia-diskimage-retrofit charmhub yoga/stable 45 ubuntu Unit is ready (Image 75e34ed11dc9 retrofitting completed at 06/04/22 03:51:24)
octavia-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
octavia-ovn-chassis 22.03.0 active 1 ovn-chassis charmhub stable 21 ubuntu Unit is ready
openstack-dashboard 22.1.0 active 1 openstack-dashboard charmhub yoga/stable 536 ubuntu Unit is ready
ovn-central 22.03.0 active 3 ovn-central charmhub 22.03/stable 31 ubuntu Unit is ready (northd
ovn-chassis 20.03.2 active 3 ovn-chassis charmhub 22.03/stable 46 ubuntu Unit is ready
placement 7.0.0 active 1 placement charmhub yoga/stable 49 ubuntu Unit is ready
placement-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
rabbitmq-server 3.8.2 active 1 rabbitmq-server charmhub 3.9/stable 149 ubuntu Unit is ready
vault 1.7.9 active 1 vault charmhub 1.7/stable 68 ubuntu Unit is ready (active
vault-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
==============================================================================
OCTAVIA ERROR LOGS:
2022-06-04 05:46:55.401 410746 INFO octavia.controller.queue.v2.endpoints [-] Creating load balancer 'e909c98a-e530-48bd-8600-b3bcdae3b04f'...
2022-06-04 05:46:55.747 410746 INFO keyring.backend [-] Loading KWallet
2022-06-04 05:46:55.755 410746 INFO keyring.backend [-] Loading SecretService
2022-06-04 05:46:55.758 410746 INFO keyring.backend [-] Loading Windows
2022-06-04 05:46:55.759 410746 INFO keyring.backend [-] Loading chainer
2022-06-04 05:46:55.759 410746 INFO keyring.backend [-] Loading macOS
2022-06-04 05:46:56.861 410746 INFO octavia.network.drivers.neutron.allowed_address_pairs [-] Port 33ef989f-1b9f-4214-9933-0d0152eabe81 already exists. Nothing to be done.
2022-06-04 05:46:56.862 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Allocated vip with port id 33ef989f-1b9f-4214-9933-0d0152eabe81, subnet id d74285aa-0765-4b25-970d-dfed2afb8d19, ip address 10.0.1.124 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:56.904 410746 INFO octavia.controller.worker.v2.tasks.database_tasks [-] Updated vip with port id 33ef989f-1b9f-4214-9933-0d0152eabe81, subnet id d74285aa-0765-4b25-970d-dfed2afb8d19, ip address 10.0.1.124 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.012 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Set up VIP SG b9d25d67-7a05-4424-a9b7-cee8cc31a055 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f complete
2022-06-04 05:46:58.112 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Got subnet d74285aa-0765-4b25-970d-dfed2afb8d19 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.171 410746 INFO octavia.controller.worker.v2.tasks.database_tasks [-] Created Amphora 76fc5262-28ea-4b8e-8483-7951578183bf in DB for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.893 410746 INFO octavia.certificates.generator.local [-] Signing a certificate request using OpenSSL locally.
2022-06-04 05:46:58.893 410746 INFO octavia.certificates.generator.local [-] Using CA Certificate from config.
2022-06-04 05:46:58.894 410746 INFO octavia.certificates.generator.local [-] Using CA Private Key from config.
2022-06-04 05:46:58.894 410746 INFO octavia.certificates.generator.local [-] Using CA Private Key Passphrase from config.
2022-06-04 05:47:01.345 410746 INFO octavia.controller.worker.v2.tasks.compute_tasks [-] Server created with id: 33506f81-44d0-40a8-bb52-cf0542b1122b for amphora id: 76fc5262-28ea-4b8e-8483-7951578183bf
2022-06-04 05:47:34.006 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb8771760>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:47:39.027 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:47:49.082 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb87cc1c0>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:47:54.101 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:48:04.158 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb87778e0>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:48:09.177 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:48:19.230 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb8777430>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:48:24.248 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.057 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' (c6c49e21-732e-499e-9834-39505935cc76) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.061 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' (a557496e-f854-42ec-8494-213c199d3357) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.064 410746 WARNING octavia.controller.worker.v2.tasks.compute_tasks [-] Reverting compute create for amphora with id 76fc5262-28ea-4b8e-8483-7951578183bf and compute id: 33506f81-44d0-40a8-bb52-cf0542b1122b
2022-06-04 06:17:49.426 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' (7f34123f-5b98-4dcb-b672-446e0c036efe) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.429 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' (b139d5c6-fe79-42e2-98c4-b43d6ab8d4b3) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.433 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-generate-serverpem' (baa9486e-5d06-4615-a617-495fdefc23ab) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.435 410746 WARNING octavia.controller.worker.v2.tasks.database_tasks [-] Reverting create amphora in DB for amp id 76fc5262-28ea-4b8e-8483-7951578183bf
2022-06-04 06:17:49.454 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' (e5a5535d-c490-4dc8-bb99-f072c279f4b5) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.457 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.GetSubnetFromVIP' (90a6e262-c75e-4d81-a828-4db3d5f46e50) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.461 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.UpdateVIPSecurityGroup' (4656aa9f-beb5-45fe-a550-7323a81a55be) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.464 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.database_tasks.UpdateVIPAfterAllocation' (0a738742-e5d9-40cc-a633-da627688f699) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.466 410746 WARNING octavia.controller.worker.v2.tasks.network_tasks [-] Deallocating vip 10.0.1.124
2022-06-04 06:17:49.467 410746 ERROR octavia.controller.worker.v2.tasks.network_tasks [-] Failed to deallocate VIP. Resources may still be in use from vip: 10.0.1.124 due to error: 'NoneType' object has no attribute 'amphorae'
2022-06-04 06:17:49.468 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.AllocateVIP' (dbfd4673-a99e-4a68-a119-5c030f0b46dd) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.471 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'reload-lb-before-allocate-vip' (e65b35f7-b6e9-4dea-9af0-13ed9b53b41d) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.483 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' (0fb07bfa-a155-4a79-a940-aa5a1b6cb34e) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.488 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Flow 'octavia-create-loadbalancer-flow' (989bd56a-d6f1-45cf-86ec-a581d96eeb18) transitioned into state 'REVERTED' from state 'RUNNING'
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server [-] Exception during message handling: octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/queue/v2/endpoints.py", line 43, in create_load_balancer
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self.worker.create_load_balancer(loadbalancer, flavor,
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 333, in wrapped_f
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self(f, *args, **kw)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 423, in __call__
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 360, in iter
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return fut.result()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self.__get_result()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise self._exception
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 426, in __call__
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/controller_worker.py", line 355, in create_load_balancer
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self.run_flow(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/controller_worker.py", line 111, in run_flow
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server tf.run()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 339, in reraise_if_any
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server failures[0].reraise()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 346, in reraise
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise value
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/tasks/amphora_driver_tasks.py", line 552, in execute
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server amp_info = self.amphora_driver.get_info(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 373, in get_info
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self._populate_amphora_api_version(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 105, in _populate_amphora_api_version
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server amphora.api_version = self.clients['base'].get_api_version(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 746, in get_api_version
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server r = self.get(amp, retry_404=False, timeout_dict=timeout_dict,
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 734, in request
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise driver_except.AmpConnectionRetry(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server |
|
| 2022-06-04 08:42:54 |
odie |
description |
Deployed openstack bundles and octavia using overlay and everything works except an error when creating Load balancer.
ERROR:
Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server
o-hm0 port doesn't have an IPv4
re deployed and set create-mgmt-network: false by default it creates IPv6 management network for Octavia and get an IP but the same error was encountered
Also tried the octavia rev 39 with same error. May I know which octavia versions are working?
========================================================================================================
Octavia bundles:
barbican - ver. 14.0.0 - rev. 67
octavia - ver - 10.0.0 rev. 71
octavia-ovn-chassis - ver. 22.03.0 - rev. 21
glance-simplestreams-sync - rev 68
octavia-diskimage-retrofit - ver 0.9.12 - rev. 45
==========================================================================================================
openstack maas-controller maas-one/default 2.9.31 unsupported 06:55:00Z
App Version Status Scale Charm Store Channel Rev OS Message
barbican 14.0.0 active 1 barbican charmhub yoga/stable 67 ubuntu Unit is ready
barbican-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
barbican-vault 3.10.0 active 1 barbican-vault charmhub stable 27 ubuntu Unit is ready
ceph-mon 15.2.16 active 3 ceph-mon charmhub quincy/stable 106 ubuntu Unit is ready and clu
ceph-osd 15.2.16 active 4 ceph-osd charmhub quincy/stable 534 ubuntu Unit is ready (1 OSD)
ceph-radosgw 17.2.0 active 1 ceph-radosgw charmhub quincy/stable 526 ubuntu Unit is ready
cinder 16.4.2 active 1 cinder charmhub yoga/stable 554 ubuntu Unit is ready
cinder-ceph 16.4.2 active 1 cinder-ceph charmhub yoga/stable 502 ubuntu Unit is ready
cinder-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
dashboard-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance 24.0.0 active 1 glance charmhub yoga/stable 544 ubuntu Unit is ready
glance-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
glance-simplestreams-sync active 1 glance-simplestreams-sync charmhub yoga/stable 68 ubuntu Unit is ready (Glance/22 06:25:28, metadata uploaded to object store)
keystone 21.0.0 active 1 keystone charmhub yoga/stable 568 ubuntu Application Ready
keystone-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
mysql-innodb-cluster 8.0.29 active 3 mysql-innodb-cluster charmhub 8.0/stable 24 ubuntu Unit is ready: Mode: nd can tolerate up to ONE failure.
neutron-api 16.4.2 active 1 neutron-api charmhub yoga/stable 526 ubuntu Unit is ready
neutron-api-plugin-ovn 16.4.2 active 1 neutron-api-plugin-ovn charmhub yoga/stable 29 ubuntu Unit is ready
neutron-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
nova-cloud-controller 21.2.4 active 1 nova-cloud-controller charmhub yoga/stable 601 ubuntu Unit is ready
nova-compute 21.2.4 active 3 nova-compute charmhub yoga/stable 588 ubuntu Unit is ready
nova-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
ntp 3.5 active 3 ntp charmhub stable 47 ubuntu chrony: Ready
octavia 10.0.0 active 1 octavia charmhub yoga/stable 71 ubuntu Unit is ready
octavia-dashboard 9.0.0 active 1 octavia-dashboard charmhub yoga/stable 44 ubuntu Unit is ready
octavia-diskimage-retrofit 0.9.12 active 1 octavia-diskimage-retrofit charmhub yoga/stable 45 ubuntu Unit is ready (Image 75e34ed11dc9 retrofitting completed at 06/04/22 03:51:24)
octavia-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
octavia-ovn-chassis 22.03.0 active 1 ovn-chassis charmhub stable 21 ubuntu Unit is ready
openstack-dashboard 22.1.0 active 1 openstack-dashboard charmhub yoga/stable 536 ubuntu Unit is ready
ovn-central 22.03.0 active 3 ovn-central charmhub 22.03/stable 31 ubuntu Unit is ready (northd
ovn-chassis 20.03.2 active 3 ovn-chassis charmhub 22.03/stable 46 ubuntu Unit is ready
placement 7.0.0 active 1 placement charmhub yoga/stable 49 ubuntu Unit is ready
placement-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
rabbitmq-server 3.8.2 active 1 rabbitmq-server charmhub 3.9/stable 149 ubuntu Unit is ready
vault 1.7.9 active 1 vault charmhub 1.7/stable 68 ubuntu Unit is ready (active
vault-mysql-router 8.0.29 active 1 mysql-router charmhub 8.0/stable 26 ubuntu Unit is ready
==============================================================================
OCTAVIA ERROR LOGS:
2022-06-04 05:46:55.401 410746 INFO octavia.controller.queue.v2.endpoints [-] Creating load balancer 'e909c98a-e530-48bd-8600-b3bcdae3b04f'...
2022-06-04 05:46:55.747 410746 INFO keyring.backend [-] Loading KWallet
2022-06-04 05:46:55.755 410746 INFO keyring.backend [-] Loading SecretService
2022-06-04 05:46:55.758 410746 INFO keyring.backend [-] Loading Windows
2022-06-04 05:46:55.759 410746 INFO keyring.backend [-] Loading chainer
2022-06-04 05:46:55.759 410746 INFO keyring.backend [-] Loading macOS
2022-06-04 05:46:56.861 410746 INFO octavia.network.drivers.neutron.allowed_address_pairs [-] Port 33ef989f-1b9f-4214-9933-0d0152eabe81 already exists. Nothing to be done.
2022-06-04 05:46:56.862 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Allocated vip with port id 33ef989f-1b9f-4214-9933-0d0152eabe81, subnet id d74285aa-0765-4b25-970d-dfed2afb8d19, ip address 10.0.1.124 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:56.904 410746 INFO octavia.controller.worker.v2.tasks.database_tasks [-] Updated vip with port id 33ef989f-1b9f-4214-9933-0d0152eabe81, subnet id d74285aa-0765-4b25-970d-dfed2afb8d19, ip address 10.0.1.124 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.012 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Set up VIP SG b9d25d67-7a05-4424-a9b7-cee8cc31a055 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f complete
2022-06-04 05:46:58.112 410746 INFO octavia.controller.worker.v2.tasks.network_tasks [-] Got subnet d74285aa-0765-4b25-970d-dfed2afb8d19 for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.171 410746 INFO octavia.controller.worker.v2.tasks.database_tasks [-] Created Amphora 76fc5262-28ea-4b8e-8483-7951578183bf in DB for load balancer e909c98a-e530-48bd-8600-b3bcdae3b04f
2022-06-04 05:46:58.893 410746 INFO octavia.certificates.generator.local [-] Signing a certificate request using OpenSSL locally.
2022-06-04 05:46:58.893 410746 INFO octavia.certificates.generator.local [-] Using CA Certificate from config.
2022-06-04 05:46:58.894 410746 INFO octavia.certificates.generator.local [-] Using CA Private Key from config.
2022-06-04 05:46:58.894 410746 INFO octavia.certificates.generator.local [-] Using CA Private Key Passphrase from config.
2022-06-04 05:47:01.345 410746 INFO octavia.controller.worker.v2.tasks.compute_tasks [-] Server created with id: 33506f81-44d0-40a8-bb52-cf0542b1122b for amphora id: 76fc5262-28ea-4b8e-8483-7951578183bf
2022-06-04 05:47:34.006 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb8771760>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:47:39.027 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:47:49.082 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb87cc1c0>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:47:54.101 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:48:04.158 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb87778e0>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:48:09.177 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 05:48:19.230 410746 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb8777430>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 05:48:24.248 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait' (0bcc97ff-41a3-49dd-a26a-b417cda4a0de) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.057 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-mark-amphora-booting-indb' (c6c49e21-732e-499e-9834-39505935cc76) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.061 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-amphora-computeid' (a557496e-f854-42ec-8494-213c199d3357) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.064 410746 WARNING octavia.controller.worker.v2.tasks.compute_tasks [-] Reverting compute create for amphora with id 76fc5262-28ea-4b8e-8483-7951578183bf and compute id: 33506f81-44d0-40a8-bb52-cf0542b1122b
2022-06-04 06:17:49.426 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-cert-compute-create' (7f34123f-5b98-4dcb-b672-446e0c036efe) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.429 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-update-cert-expiration' (b139d5c6-fe79-42e2-98c4-b43d6ab8d4b3) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.433 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-generate-serverpem' (baa9486e-5d06-4615-a617-495fdefc23ab) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.435 410746 WARNING octavia.controller.worker.v2.tasks.database_tasks [-] Reverting create amphora in DB for amp id 76fc5262-28ea-4b8e-8483-7951578183bf
2022-06-04 06:17:49.454 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-create-amphora-indb' (e5a5535d-c490-4dc8-bb99-f072c279f4b5) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.457 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.GetSubnetFromVIP' (90a6e262-c75e-4d81-a828-4db3d5f46e50) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.461 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.UpdateVIPSecurityGroup' (4656aa9f-beb5-45fe-a550-7323a81a55be) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.464 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.database_tasks.UpdateVIPAfterAllocation' (0a738742-e5d9-40cc-a633-da627688f699) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.466 410746 WARNING octavia.controller.worker.v2.tasks.network_tasks [-] Deallocating vip 10.0.1.124
2022-06-04 06:17:49.467 410746 ERROR octavia.controller.worker.v2.tasks.network_tasks [-] Failed to deallocate VIP. Resources may still be in use from vip: 10.0.1.124 due to error: 'NoneType' object has no attribute 'amphorae'
2022-06-04 06:17:49.468 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.network_tasks.AllocateVIP' (dbfd4673-a99e-4a68-a119-5c030f0b46dd) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.471 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'reload-lb-before-allocate-vip' (e65b35f7-b6e9-4dea-9af0-13ed9b53b41d) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.483 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Task 'octavia.controller.worker.v2.tasks.lifecycle_tasks.LoadBalancerIDToErrorOnRevertTask' (0fb07bfa-a155-4a79-a940-aa5a1b6cb34e) transitioned into state 'REVERTED' from state 'REVERTING'
2022-06-04 06:17:49.488 410746 WARNING octavia.controller.worker.v2.controller_worker [-] Flow 'octavia-create-loadbalancer-flow' (989bd56a-d6f1-45cf-86ec-a581d96eeb18) transitioned into state 'REVERTED' from state 'RUNNING'
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server [-] Exception during message handling: octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/queue/v2/endpoints.py", line 43, in create_load_balancer
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self.worker.create_load_balancer(loadbalancer, flavor,
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 333, in wrapped_f
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self(f, *args, **kw)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 423, in __call__
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 360, in iter
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return fut.result()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server return self.__get_result()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise self._exception
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 426, in __call__
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/controller_worker.py", line 355, in create_load_balancer
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self.run_flow(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/controller_worker.py", line 111, in run_flow
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server tf.run()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 339, in reraise_if_any
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server failures[0].reraise()
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 346, in reraise
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise value
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/controller/worker/v2/tasks/amphora_driver_tasks.py", line 552, in execute
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server amp_info = self.amphora_driver.get_info(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 373, in get_info
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server self._populate_amphora_api_version(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 105, in _populate_amphora_api_version
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server amphora.api_version = self.clients['base'].get_api_version(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 746, in get_api_version
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server r = self.get(amp, retry_404=False, timeout_dict=timeout_dict,
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py", line 734, in request
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server raise driver_except.AmpConnectionRetry(
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server octavia.amphorae.driver_exceptions.exceptions.AmpConnectionRetry: Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server |
Greetings Charmhub!
I've deployed charmed ocatavia overlay bundles to provide load balancing functionality so that I can deploy charmed Kubernetes clusters on Openstack.
All the resources are deployed and works fine but when trying to create load balancer I've encountered this errors.
ERROR:
Could not connect to amphora, exception caught: HTTPSConnectionPool(host='172.31.0.75', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7efdb88eab50>, 'Connection to 172.31.0.75 timed out. (connect timeout=10.0)'))
2022-06-04 06:17:49.489 410746 ERROR oslo_messaging.rpc.server
=========================================
o-hm0 port doesn't have an IPv4
- re-deployed octavia and set create-mgmt-network: false
to allow pre-configure octavia management network
IPv6 management network for Octavia gets an IP but the same error was encountered
May I know which octavia versions are working with openstack xena or yoga?
===============================================================================
Octavia bundles:
barbican - ver. 14.0.0 - rev. 67
octavia - ver - 10.0.0 rev. 71
octavia-ovn-chassis - ver. 22.03.0 - rev. 21
glance-simplestreams-sync - rev 68
octavia-diskimage-retrofit - ver 0.9.12 - rev. 45
Here's a dump of loctavia load balancer error logs and my Openstack Network/port configuration:
https://pastebin.ubuntu.com/p/9ScrywZfWG/ |
|
