I have tried to switch it to valid_interfaces prior, but it did not work. I
also tried to patch code myself, but did not succeed as I don't understand
how Octavia works fully.

[glance]
region_name = 5Net
endpoint_type = internal
ca_certificates_file = /etc/ssl/certs/ca-certificates.crt

[neutron]
region_name = 5Net
valid_interfaces = internal
ca_certificates_file = /etc/ssl/certs/ca-certificates.crt
endpoint_override = https://cloud.5net.lan.tunninet.com:9696

[nova]
region_name = 5Net
endpoint_type = internal
ca_certificates_file = /etc/ssl/certs/ca-certificates.crt
availability_zone = 5Net

[cinder]
availability_zone = 5Net
endpoint_type = internal
ᐧ

On Mon, Apr 15, 2024 at 3:25 AM Gregory Thiemonge <
<email address hidden>> wrote:

> Can you share the [nova] [cinder] [glance] sections of your octavia
> configuration?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2061268
>
> Title:
>   2023.2 Code Patched with:
>
> https://opendev.org/openstack/octavia/commit/7bb6096eccc7966bd963a0529dc7b352246dfdbb
>   - can not create a new LB
>
> Status in octavia:
>   New
>
> Bug description:
>   root@5net:~# tail -f /var/log/kolla/octavia/octavia-*.log
>   ==> /var/log/kolla/octavia/octavia-api-access.log <==
>   192.168.5.99 - - [14/Apr/2024:02:11:05 -0400] "GET
> /v2.0/octavia/amphorae HTTP/1.1" 200 38 14403 "-" "gophercloud/2.0.0"
>   - - - [14/Apr/2024:02:11:05 -0400] "GET / HTTP/1.1" 200 3978 2490 "-"
> "curl-healthcheck"
>   192.168.5.99 - - [14/Apr/2024:02:11:09 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 35714 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:14 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 34457 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:15 -0400] "GET
> /v2.0/lbaas/loadbalancers HTTP/1.1" 200 48 61885 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:15 -0400] "GET
> /v2.0/octavia/amphorae HTTP/1.1" 200 38 15925 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:19 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 80599 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:24 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 80288 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:25 -0400] "GET
> /v2.0/lbaas/loadbalancers HTTP/1.1" 200 48 122145 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:25 -0400] "GET
> /v2.0/octavia/amphorae HTTP/1.1" 200 38 13787 "-" "gophercloud/2.0.0"
>
>   ==> /var/log/kolla/octavia/octavia-api-error.log <==
>   2024-04-14 02:02:45.880226   warnings.warn(deprecated_msg)
>   2024-04-14 02:02:45.880255
> /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_policy/policy.py:810:
> UserWarning: Policy
> "load-balancer:member_and_owner":"role:load-balancer_member and
> rule:load-balancer:owner" was deprecated in W in favor of
> "load-balancer:member_and_owner":"role:load-balancer_member and
> rule:project-member". Reason: The Octavia API now requires the OpenStack
> default roles and scoped tokens. See
> https://docs.openstack.org/octavia/latest/configuration/policy.html and
> https://docs.openstack.org/keystone/latest/contributor/services.html#reusable-default-roles
> for more information.. Either ensure your deployment is ready for the new
> default or copy/paste the deprecated policy into your policy file and
> maintain it manually.
>   2024-04-14 02:02:45.880260   warnings.warn(deprecated_msg)
>   2024-04-14 02:02:59.447984 Deprecated: Option "ca_certificates_file"
> from group "neutron" is deprecated for removal (The cafile option defined
> by keystoneauth1 is the new name for this option.).  Its value may be
> silently ignored in the future.
>   2024-04-14 02:03:00.082530
> /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_policy/policy.py:810:
> UserWarning: Policy "context_is_admin":"role:admin or
> role:load-balancer_admin" was deprecated in W in favor of
> "context_is_admin":"role:load-balancer_admin or rule:system-admin or
> role:admin". Reason: The Octavia API now requires the OpenStack default
> roles and scoped tokens. See
> https://docs.openstack.org/octavia/latest/configuration/policy.html and
> https://docs.openstack.org/keystone/latest/contributor/services.html#reusable-default-roles
> for more information.. Either ensure your deployment is ready for the new
> default or copy/paste the deprecated policy into your policy file and
> maintain it manually.
>   2024-04-14 02:03:00.082554   warnings.warn(deprecated_msg)
>   2024-04-14 02:03:00.082595
> /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_policy/policy.py:810:
> UserWarning: Policy
> "load-balancer:observer_and_owner":"role:load-balancer_observer and
> rule:load-balancer:owner" was deprecated in W in favor of
> "load-balancer:observer_and_owner":"role:load-balancer_observer and
> rule:project-reader". Reason: The Octavia API now requires the OpenStack
> default roles and scoped tokens. See
> https://docs.openstack.org/octavia/latest/configuration/policy.html and
> https://docs.openstack.org/keystone/latest/contributor/services.html#reusable-default-roles
> for more information.. Either ensure your deployment is ready for the new
> default or copy/paste the deprecated policy into your policy file and
> maintain it manually.
>   2024-04-14 02:03:00.082614   warnings.warn(deprecated_msg)
>   2024-04-14 02:03:00.082643
> /var/lib/kolla/venv/lib/python3.10/site-packages/oslo_policy/policy.py:810:
> UserWarning: Policy
> "load-balancer:member_and_owner":"role:load-balancer_member and
> rule:load-balancer:owner" was deprecated in W in favor of
> "load-balancer:member_and_owner":"role:load-balancer_member and
> rule:project-member". Reason: The Octavia API now requires the OpenStack
> default roles and scoped tokens. See
> https://docs.openstack.org/octavia/latest/configuration/policy.html and
> https://docs.openstack.org/keystone/latest/contributor/services.html#reusable-default-roles
> for more information.. Either ensure your deployment is ready for the new
> default or copy/paste the deprecated policy into your policy file and
> maintain it manually.
>   2024-04-14 02:03:00.082649   warnings.warn(deprecated_msg)
>
>   ==> /var/log/kolla/octavia/octavia-api.log <==
>   2024-04-14 02:02:59.558 733 INFO octavia.api.app [-]
> oslo_messaging_notifications.topics = ['notifications']
>   2024-04-14 02:02:59.558 733 INFO octavia.api.app [-]
> oslo_messaging_notifications.transport_url = ****
>   2024-04-14 02:02:59.558 733 INFO octavia.api.app [-]
> ********************************************************************************
>   2024-04-14 02:02:59.759 733 WARNING keystonemiddleware.auth_token [-]
> AuthToken middleware is set with
> keystone_authtoken.service_token_roles_required set to False. This is
> backwards compatible but deprecated behaviour. Please set this to True.
>   2024-04-14 02:09:18.979 735 WARNING openstack [None
> req-dd0763f2-f57d-49ef-b956-aab069076211 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
>   2024-04-14 02:09:18.979 735 WARNING openstack [None
> req-dd0763f2-f57d-49ef-b956-aab069076211 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
>   2024-04-14 02:09:18.980 735 WARNING openstack [None
> req-dd0763f2-f57d-49ef-b956-aab069076211 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
>   2024-04-14 02:10:11.649 735 WARNING openstack [None
> req-3a997194-acf1-4f44-9916-ea04569c7b6f - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
>   2024-04-14 02:10:11.649 735 WARNING openstack [None
> req-3a997194-acf1-4f44-9916-ea04569c7b6f - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
>   2024-04-14 02:10:11.650 735 WARNING openstack [None
> req-3a997194-acf1-4f44-9916-ea04569c7b6f - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
>
>   ==> /var/log/kolla/octavia/octavia-health-manager.log <==
>   2024-04-14 01:44:51.574 7 INFO octavia.cmd.health_manager [-] Health
> manager check process starts:
>   2024-04-14 01:44:51.576 1008 INFO
> octavia.amphorae.drivers.health.heartbeat_udp [-] attempting to listen on
> 192.168.5.1 port 5555
>   2024-04-14 01:59:43.557 7 INFO octavia.cmd.health_manager [-] Health
> Manager exiting due to signal
>   2024-04-14 01:59:44.406 1008 INFO octavia.cmd.health_manager [-] Waiting
> for executor to shutdown...
>   2024-04-14 01:59:44.407 1008 INFO octavia.cmd.health_manager [-]
> Executor shutdown finished.
>   2024-04-14 02:02:43.810 7 INFO octavia.common.config [-] Logging enabled!
>   2024-04-14 02:02:43.810 7 INFO octavia.common.config [-]
> /var/lib/kolla/venv/bin/octavia-health-manager version 13.0.1.dev20
>   2024-04-14 02:02:43.856 7 INFO octavia.cmd.health_manager [-] Health
> Manager listener process starts:
>   2024-04-14 02:02:43.860 7 INFO octavia.cmd.health_manager [-] Health
> manager check process starts:
>   2024-04-14 02:02:43.861 731 INFO
> octavia.amphorae.drivers.health.heartbeat_udp [-] attempting to listen on
> 192.168.5.1 port 5555
>
>   ==> /var/log/kolla/octavia/octavia-housekeeping.log <==
>   2024-04-14 01:44:52.907 7 INFO octavia.cmd.house_keeping [-] Load
> balancer expiry age is 604800 seconds
>   2024-04-14 02:02:40.900 7 INFO octavia.cmd.house_keeping [-] Attempting
> to gracefully terminate House-Keeping
>   2024-04-14 02:02:40.901 7 INFO octavia.cmd.house_keeping [-]
> House-Keeping process terminated
>   2024-04-14 02:02:45.736 7 INFO octavia.common.config [-] Logging enabled!
>   2024-04-14 02:02:45.736 7 INFO octavia.common.config [-]
> /var/lib/kolla/venv/bin/octavia-housekeeping version 13.0.1.dev20
>   2024-04-14 02:02:45.839 7 INFO octavia.cmd.house_keeping [-] Starting
> house keeping at 2024-04-14 06:02:45.839614
>   2024-04-14 02:02:45.840 7 INFO octavia.cmd.house_keeping [-] DB cleanup
> interval is set to 30 sec
>   2024-04-14 02:02:45.840 7 INFO octavia.cmd.house_keeping [-] Amphora
> expiry age is 604800 seconds
>   2024-04-14 02:02:45.840 7 INFO octavia.cmd.house_keeping [-] Load
> balancer expiry age is 604800 seconds
>   2024-04-14 02:02:45.840 7 INFO octavia.cmd.house_keeping [-] Expiring
> certificate check interval is set to 3600 sec
>
>   ==> /var/log/kolla/octavia/octavia-worker.log <==
>   2024-04-14 01:45:01.876 731 INFO octavia.controller.queue.v2.consumer
> [-] Starting V2 consumer...
>   2024-04-14 02:02:47.138 7 INFO cotyledon._service_manager [-] Caught
> SIGTERM signal, graceful exiting of master process
>   2024-04-14 02:02:47.239 731 INFO cotyledon._service [-] Caught SIGTERM
> signal, graceful exiting of service ConsumerService(0) [731]
>   2024-04-14 02:02:47.239 731 INFO octavia.controller.queue.v2.consumer
> [-] Stopping V2 consumer...
>   2024-04-14 02:02:51.980 731 INFO octavia.controller.queue.v2.consumer
> [-] V2 Consumer successfully stopped.  Waiting for final messages to be
> processed...
>   2024-04-14 02:02:51.985 731 WARNING amqp [-] Received method (60, 30)
> during closing channel 1. This method will be ignored
>   2024-04-14 02:02:51.986 731 INFO octavia.controller.queue.v2.consumer
> [-] Shutting down V2 endpoint worker executors...
>   2024-04-14 02:02:56.757 7 INFO octavia.common.config [-] Logging enabled!
>   2024-04-14 02:02:56.757 7 INFO octavia.common.config [-]
> /var/lib/kolla/venv/bin/octavia-worker version 13.0.1.dev20
>   2024-04-14 02:02:56.843 731 INFO octavia.controller.queue.v2.consumer
> [-] Starting V2 consumer...
>
>   ==> /var/log/kolla/octavia/octavia-api-access.log <==
>   192.168.5.99 - - [14/Apr/2024:02:11:29 -0400] "GET
> /v2.0/lbaas/availabilityzones HTTP/1.1" 200 58 39504 "-"
> "openstacksdk/1.5.0 keystoneauth1/5.3.0 python-requests/2.28.2
> CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:29 -0400] "GET /v2.0/lbaas/flavors
> HTTP/1.1" 200 36 96963 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:29 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 37895 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:34 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 38753 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>   192.168.5.99 - - [14/Apr/2024:02:11:35 -0400] "GET
> /v2.0/lbaas/loadbalancers HTTP/1.1" 200 48 149652 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:35 -0400] "GET
> /v2.0/octavia/amphorae HTTP/1.1" 200 38 12304 "-" "gophercloud/2.0.0"
>   - - - [14/Apr/2024:02:11:35 -0400] "GET / HTTP/1.1" 200 3978 2702 "-"
> "curl-healthcheck"
>   192.168.5.99 - - [14/Apr/2024:02:11:36 -0400] "GET
> /v2.0/lbaas/loadbalancers HTTP/1.1" 200 48 58485 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:36 -0400] "GET
> /v2.0/octavia/amphorae HTTP/1.1" 200 38 17650 "-" "gophercloud/2.0.0"
>   192.168.5.99 - - [14/Apr/2024:02:11:39 -0400] "GET
> /v2.0/lbaas/loadbalancers?project_id=e95c8c84b62e416daf03b91a9648962b
> HTTP/1.1" 200 48 36406 "-" "openstacksdk/1.5.0 keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>
>   ==> /var/log/kolla/octavia/octavia-api.log <==
>   2024-04-14 02:11:42.134 736 WARNING openstack [None
> req-f4cdd978-d9ce-4cd3-ba91-6fff2d85ea99 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
>   2024-04-14 02:11:42.134 736 WARNING openstack [None
> req-f4cdd978-d9ce-4cd3-ba91-6fff2d85ea99 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
>   2024-04-14 02:11:42.134 736 WARNING openstack [None
> req-f4cdd978-d9ce-4cd3-ba91-6fff2d85ea99 - e95c8c84b62e416daf03b91a9648962b
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
>
>
>   Also, separate issue - prior to this, i had additional patches from that
> had it up but when i rebooted, it totally lost state and was not
> recoverable. It sat in a pending state forever.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/octavia/+bug/2061268/+subscriptions
>
>