OK, an update ... The above config changing everything worked with the
above errors tho?!

Here is what happens when i do not update the new moniker for glance,
cinder, nova etc.... (which makes no sense to me?)

==> /var/log/kolla/octavia/octavia-api.log <==
2024-02-01 12:36:03.842 733 WARNING openstack [None
req-366da871-ab3b-4e98-9780-0f29a93cfe14 - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'block-storage': Encountered an
exception attempting to process config for project 'cinder' (service type
'block-storage'): no such option valid_interfaces in group [cinder]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[cinder]
2024-02-01 12:36:03.843 733 WARNING openstack [None
req-366da871-ab3b-4e98-9780-0f29a93cfe14 - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'compute': Encountered an exception
attempting to process config for project 'nova' (service type 'compute'):
no such option valid_interfaces in group [nova]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[nova]
2024-02-01 12:36:03.843 733 WARNING openstack [None
req-366da871-ab3b-4e98-9780-0f29a93cfe14 - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'image': Encountered an exception
attempting to process config for project 'glance' (service type 'image'):
no such option valid_interfaces in group [glance]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[glance]
2024-02-01 12:36:03.884 737 WARNING openstack [None
req-4b65bd1b-3e6d-437d-a879-c08a1eaf225a - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'block-storage': Encountered an
exception attempting to process config for project 'cinder' (service type
'block-storage'): no such option valid_interfaces in group [cinder]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[cinder]
2024-02-01 12:36:03.884 737 WARNING openstack [None
req-4b65bd1b-3e6d-437d-a879-c08a1eaf225a - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'compute': Encountered an exception
attempting to process config for project 'nova' (service type 'compute'):
no such option valid_interfaces in group [nova]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[nova]
2024-02-01 12:36:03.885 737 WARNING openstack [None
req-4b65bd1b-3e6d-437d-a879-c08a1eaf225a - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Disabling service 'image': Encountered an exception
attempting to process config for project 'glance' (service type 'image'):
no such option valid_interfaces in group [glance]:
oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
[glance]
2024-02-01 12:36:03.994 733 INFO octavia.api.v2.controllers.member [None
req-366da871-ab3b-4e98-9780-0f29a93cfe14 - e5b9296fbd9e4d9ea5e925780c64690f
- - default default] Sending create Member
4a551e43-42b8-49ea-a989-42a28c1466ef to provider amphora


Why when i dont provide this is it saying this?

Unrelated, I also see this:

I also saw this Error in the logs:


==> /var/log/kolla/octavia/octavia-api-error.log <==
2024-02-01 12:14:01.370602
/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/db/repositories.py:1017:
SAWarning: Coercing Subquery object into a select() for use in IN(); please
pass a select() construct explicitly
2024-02-01 12:14:01.370654
filter(models.LoadBalancer.id.in_(lb_subquery)).first())
2024-02-01 12:14:01.373214
/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/db/repositories.py:1017:
SAWarning: SELECT statement has a cartesian product between FROM element(s)
"pool" and FROM element "load_balancer".  Apply join condition(s) between
each element to resolve.
2024-02-01 12:14:01.373248
filter(models.LoadBalancer.id.in_(lb_subquery)).first())


Also, what about [certificates]?

endpoint_type = internal
or
valid_interfaces = internal
?


ᐧ

On Thu, Feb 1, 2024 at 12:19 PM Noel Ashford <email address hidden> wrote:

> Did any other sections like NOVA etc change as well? Should all instances
> of endpoint_type be removed from octavia or just Neutron?
>
> I updated all sections to use the new monikers in one attempt and only
> [neutron] updated in the second attempt.
>
> As it complained about the below, I tried to update [glance], [nova]
> [cinder] and still it gives me the same ?
>
> It got further but seemingly broke on the other services it needs
> regardless of the above approaches.
>
> [image: image.png]
>
>
>
> ==> /var/log/kolla/octavia/octavia-worker.log <==
> 2024-02-01 12:12:56.958 731 WARNING octavia.common.base_taskflow [-] Task
> 'STANDALONE-octavia-create-amp-for-lb-subflow-octavia-amp-compute-connectivity-wait'
> (1747f037-6b94-448b-98f5-e2fad599f03d) transitioned into state 'REVERTED'
> from state 'REVERTING'
> 2024-02-01 12:12:57.134 731 WARNING
> octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to
> instance. Retrying.: requests.exceptions.ConnectionError:
> HTTPSConnectionPool(host='172.16.5.129', port=9443): Max retries exceeded
> with url: // (Caused by
> NewConnectionError('<urllib3.connection.HTTPSConnection object at
> 0x7f1b9057fa60>: Failed to establish a new connection: [Errno 111]
> Connection refused'))
>
> ==> /var/log/kolla/octavia/octavia-api-access.log <==
> 2024-02-01 12:12:32.207 733 WARNING openstack [None
> req-1ee9dfa6-a33d-4550-951a-d05260daa944 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
> 2024-02-01 12:12:32.208 733 WARNING openstack [None
> req-1ee9dfa6-a33d-4550-951a-d05260daa944 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
> 2024-02-01 12:12:32.208 733 WARNING openstack [None
> req-1ee9dfa6-a33d-4550-951a-d05260daa944 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
> 2024-02-01 12:12:32.242 736 WARNING openstack [None
> req-5bb468c8-0f06-4351-ab5f-4b5754d62775 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
> 2024-02-01 12:12:32.242 736 WARNING openstack [None
> req-5bb468c8-0f06-4351-ab5f-4b5754d62775 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
> 2024-02-01 12:12:32.242 736 WARNING openstack [None
> req-5bb468c8-0f06-4351-ab5f-4b5754d62775 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
> 2024-02-01 12:12:32.917 733 INFO
> octavia.api.drivers.amphora_driver.v2.driver [None
> req-1ee9dfa6-a33d-4550-951a-d05260daa944 - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Amphora provider created VIP port
> 94aa1bdc-a3f5-422b-a4af-7a72a2696175 for load balancer
> 127ec943-3ac6-46c0-b9ec-eca28aafaed4.
> 2024-02-01 12:12:32.921 733 INFO octavia.api.v2.controllers.load_balancer
> [None req-1ee9dfa6-a33d-4550-951a-d05260daa944 -
> e5b9296fbd9e4d9ea5e925780c64690f - - default default] Sending create Load
> Balancer 127ec943-3ac6-46c0-b9ec-eca28aafaed4 to provider amphora
>
>
> Additional Info
>
>
> ==> /var/log/kolla/octavia/octavia-api.log <==
> 2024-02-01 12:14:11.449 736 WARNING openstack [None
> req-3db6d72d-420c-47e0-9420-7ef25282058e - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
> 2024-02-01 12:14:11.450 736 WARNING openstack [None
> req-3db6d72d-420c-47e0-9420-7ef25282058e - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
> 2024-02-01 12:14:11.451 736 WARNING openstack [None
> req-3db6d72d-420c-47e0-9420-7ef25282058e - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
> 2024-02-01 12:14:11.625 736 INFO octavia.api.v2.controllers.member [None
> req-3db6d72d-420c-47e0-9420-7ef25282058e - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Sending create Member
> b385a83d-8f36-4a0d-b801-199351b3172b to provider amphora
>
> ==> /var/log/kolla/octavia/octavia-worker.log <==
> 2024-02-01 12:14:11.684 731 INFO octavia.controller.queue.v2.endpoints [-]
> Creating member 'b385a83d-8f36-4a0d-b801-199351b3172b'...
>
> ==> /var/log/kolla/octavia/octavia-api-access.log <==
> 192.168.5.99 - - [01/Feb/2024:12:14:11 -0500] "POST
> /v2.0/lbaas/pools/b1d75929-cfdc-4f7e-92b2-f5b38ef5be4c/members HTTP/1.1"
> 201 513 619189 "-" "heat-engine keystoneauth1/5.3.0 python-requests/2.28.2
> CPython/3.10.12"
> 192.168.5.99 - - [01/Feb/2024:12:14:11 -0500] "GET
> /v2.0/lbaas/pools/b1d75929-cfdc-4f7e-92b2-f5b38ef5be4c/members/b385a83d-8f36-4a0d-b801-199351b3172b
> HTTP/1.1" 200 513 95692 "-" "heat-engine keystoneauth1/5.3.0
> python-requests/2.28.2 CPython/3.10.12"
>
> ==> /var/log/kolla/octavia/octavia-api.log <==
> 2024-02-01 12:14:12.309 735 WARNING openstack [None
> req-8dfeb88b-3ce9-46e7-bebf-01079cc1819b - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'block-storage': Encountered an
> exception attempting to process config for project 'cinder' (service type
> 'block-storage'): no such option valid_interfaces in group [cinder]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [cinder]
> 2024-02-01 12:14:12.310 735 WARNING openstack [None
> req-8dfeb88b-3ce9-46e7-bebf-01079cc1819b - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'compute': Encountered an exception
> attempting to process config for project 'nova' (service type 'compute'):
> no such option valid_interfaces in group [nova]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [nova]
> 2024-02-01 12:14:12.311 735 WARNING openstack [None
> req-8dfeb88b-3ce9-46e7-bebf-01079cc1819b - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Disabling service 'image': Encountered an exception
> attempting to process config for project 'glance' (service type 'image'):
> no such option valid_interfaces in group [glance]:
> oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group
> [glance]
> 2024-02-01 12:14:12.495 735 INFO octavia.api.v2.controllers.member [None
> req-8dfeb88b-3ce9-46e7-bebf-01079cc1819b - e5b9296fbd9e4d9ea5e925780c64690f
> - - default default] Sending create Member
> 7cd72bca-c140-4ab0-a081-7c8db5b94033 to provider amphora
>
>
> ᐧ
>
> On Thu, Feb 1, 2024 at 4:08 AM Gregory Thiemonge <
> <email address hidden>> wrote:
>
>> --
>>
>> [neutron]
>> region_name = TN_DEV_NY_5_NET
>> endpoint_type = internal
>> ca_certificates_file = /etc/ssl/certs/ca-certificates.crt
>> endpoint = https://int.noel.openstack.tunninet.com:9696
>>
>> --
>>
>> In 2023.2, we have deprecated the `endpoint_type` and `endpoint`
>> parameters in favor of `valid_interfaces` and `endpoint_override`.
>> we added some code to convert automatically the "old" settings to the new
>> ones, but I think there's a bug (I'm reproducing a similar issue in my env).
>>
>> I found a possible workaround, if you rename the keys of the settings
>> with the new names (only in the neutron section), it should work
>> correctly:
>>
>> [neutron]
>> region_name = TN_DEV_NY_5_NET
>> valid_interfaces = internal
>> ca_certificates_file = /etc/ssl/certs/ca-certificates.crt
>> endpoint_override = https://int.noel.openstack.tunninet.com:9696
>>
>> can you try that?
>>
>> On my side, I'm working on a bugfix.
>>
>> ** Changed in: octavia
>>    Importance: Undecided => High
>>
>> ** Changed in: octavia
>>        Status: New => Confirmed
>>
>> ** Changed in: octavia
>>      Assignee: (unassigned) => Gregory Thiemonge (gthiemonge)
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/2051604
>>
>> Title:
>>   Magnum -> Octavia, Neutron driver - Not using region or availability
>>   zone to search, rather picking a random neutron endpoint causing
>>   lookup failures on subnets
>>
>> Status in octavia:
>>   Confirmed
>>
>> Bug description:
>>   t is lookup up on the wrong endpoint... I check endpoints, no errors
>>   there - seems there is no filter to tell it what to look at.... 2023.2
>>   is my version. I never saw this issue prior to 2023.x & amphorav2
>>
>>   Trigger = Call Magnum COE create, it creates a LB, looks up the subnet
>>   and chooses the wrong endpoint when the LB is being made in the
>>   neutron driver for octavia....
>>
>>   2024-01-29 17:15:55.984 733 ERROR wsme.api [None
>> req-026ef7fd-67aa-412a-89e1-a7529c45e72c - e5b9296fbd9e4d9ea5e925780c64690f
>> - - default default] Server-side error: "subnet not found (subnet id:
>> 1b98d0d6-3eaa-490d-a9a4-e351fb0ebedf).". Detail:
>>   Traceback (most recent call last):
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/network/drivers/neutron/base.py",
>> line 189, in _get_resource
>>       resource = getattr(
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/network/v2/_proxy.py",
>> line 5111, in get_subnet
>>       return self._get(_subnet.Subnet, subnet)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/proxy.py", line
>> 61, in check
>>       return method(self, expected, actual, *args, **kwargs)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/proxy.py", line
>> 665, in _get
>>       return res.fetch(
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/resource.py",
>> line 1711, in fetch
>>       self._translate_response(response, **kwargs)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/resource.py",
>> line 1287, in _translate_response
>>       exceptions.raise_from_response(response,
>> error_message=error_message)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/openstack/exceptions.py",
>> line 250, in raise_from_response
>>       raise cls(
>>
>>   openstack.exceptions.ResourceNotFound: No Subnet found for
>>   1b98d0d6-3eaa-490d-a9a4-e351fb0ebedf: Client Error for url:
>>
>> https://int.dave.openstack.tunninet.com:9696/v2.0/subnets/1b98d0d6-3eaa-490d-a9a4-e351fb0ebedf
>> ,
>>   Subnet 1b98d0d6-3eaa-490d-a9a4-e351fb0ebedf could not be found.
>>
>>   The above exception was the direct cause of the following exception:
>>
>>   Traceback (most recent call last):
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/wsmeext/pecan.py", line
>> 82, in callfunction
>>       result = f(self, *args, **kwargs)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/api/v2/controllers/load_balancer.py",
>> line 453, in post
>>       self._validate_vip_request_object(load_balancer, context=context)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/api/v2/controllers/load_balancer.py",
>> line 308, in _validate_vip_request_object
>>
>> self._validate_subnets_share_network_but_no_duplicates(load_balancer)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/api/v2/controllers/load_balancer.py",
>> line 243, in _validate_subnets_share_network_but_no_duplicates
>>       used_subnets[subnet_id] = network_driver.get_subnet(subnet_id)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/network/drivers/neutron/base.py",
>> line 250, in get_subnet
>>       return self._get_resource('subnet', subnet_id, context=context)
>>
>>     File
>> "/var/lib/kolla/venv/lib/python3.10/site-packages/octavia/network/drivers/neutron/base.py",
>> line 197, in _get_resource
>>       raise getattr(base, '%sNotFound' % ''.join(
>>
>>
>>
>> https://docs.openstack.org/octavia/latest/_modules/octavia/network/drivers/neutron/base.html#BaseNeutronDriver.get_subnet
>>
>>   def _get_resources_by_filters(self, resource_type, unique_item=False,
>>                                     **filters):
>>           """Retrieves item(s) from filters. By default, a list is
>> returned.
>>
>>           If unique_item set to True, only the first resource is returned.
>>           """
>>           try:
>>               resources = getattr(
>>                   self.network_proxy, f"{resource_type}s")(**filters)
>>               conversion_function = getattr(
>>                   utils,
>>                   'convert_%s_to_model' % resource_type)
>>               try:
>>                   # get first item to see if there is at least one
>> resource
>>                   res_list = [conversion_function(next(resources))]
>>               except StopIteration:
>>                   # pylint: disable=raise-missing-from
>>                   raise os_exceptions.NotFoundException(
>>                       f'No resource of type {resource_type} found that
>> matches '
>>                       f'given filter criteria: {filters}.')
>>
>>               if unique_item:
>>                   return res_list[0]
>>               return res_list + [conversion_function(r) for r in
>> resources]
>>
>>           except os_exceptions.NotFoundException as e:
>>               message = _('{resource_type} not found '
>>                           '({resource_type} Filters: {filters}.').format(
>>                   resource_type=resource_type, filters=filters)
>>               raise getattr(base, '%sNotFound' % ''.join(
>>                   [w.capitalize() for w in resource_type.split('_')]
>>               ))(message) from e
>>           except Exception as e:
>>               message = _('Error retrieving {resource_type} '
>>                           '({resource_type} Filters: {filters}.').format(
>>                   resource_type=resource_type, filters=filters)
>>               LOG.exception(message)
>>               raise base.NetworkException(message) from e
>>
>>   [docs]
>>       def get_network(self, network_id, context=None):
>>           return self._get_resource('network', network_id,
>> context=context)
>>
>>   [docs]
>>       def get_subnet(self, subnet_id, context=None):
>>           return self._get_resource('subnet', subnet_id, context=context)
>>
>>   Somewhere, it is not using context to create the correct filter.... I
>>   am dead in the water on octavia usage without this, its also impacting
>>   magnum as i using octavia for this ;0 Thoughts ?
>>
>>   cross posting here: https://bugs.launchpad.net/kolla-
>>   ansible/+bug/2051602
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/octavia/+bug/2051604/+subscriptions
>>
>>