oslo.limit

_get_endpoint_by_service_lookup() fails with TypeError: object of type 'generator' has no len()

Bug #2123895 reported by melanie witt
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.limit
Fix Released
Critical
Takashi Kajinami

Bug Description

Recently tried configuring [oslo_limit] without endpoint_id in order to use the endpoint lookup mechanism and found that it fails due to openstacksdk returning a generator for lists of resources. When oslo.limit tries to use len() on the generator, it fails as the generator must be converted to a list before len() could be used.

This is an example traceback from a nova-next job run [1] on a proposed devstack patch [2]:

ep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 767, in wrapped
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 217, in wrapper
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 217, in wrapper
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 217, in wrapper
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi [Previous line repeated 11 more times]
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 783, in create
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create(
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.564565 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2246, in create
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return self._create_instance(
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1773, in _create_instance
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances(
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1412, in _provision_instances
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi num_instances = placement_limits.enforce_num_instances_and_flavor(
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/limit/placement.py", line 176, in enforce_num_instances_and_flavor
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi enforcer = enforcer or _get_enforcer(context, project_id)
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/limit/placement.py", line 149, in _get_enforcer
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return limit.Enforcer(callback)
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 86, in __init__
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi self.model = self._get_model_impl(usage_callback, cache=cache)
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 97, in _get_model_impl
Sep 16 01:11:17.566048 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return impl(usage_callback, cache=cache)
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 209, in __init__
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi self._utils = _EnforcerUtils(cache=cache)
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 276, in __init__
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi self._endpoint = self._get_endpoint()
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 285, in _get_endpoint
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi return self._get_endpoint_by_service_lookup()
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_limit/limit.py", line 308, in _get_endpoint_by_service_lookup
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi if len(services) > 1:
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi ^^^^^^^^^^^^^
Sep 16 01:11:17.567315 np68e197ffa7fe4 <email address hidden>[76526]: ERROR nova.api.openstack.wsgi TypeError: object of type 'generator' has no len()

[1] https://zuul.opendev.org/t/openstack/build/0cbb1b87e7434abc879a693e2ef23965/log/controller/logs/screen-n-api.txt
[2] https://review.opendev.org/c/openstack/devstack/+/961326

Takashi Kajinami (kajinamit)
Changed in oslo.limit:
assignee: nobody → Takashi Kajinami (kajinamit)
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.limit (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/oslo.limit/+/961329

Changed in oslo.limit:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.limit (master)

Reviewed: https://review.opendev.org/c/openstack/oslo.limit/+/961329
Committed: https://opendev.org/openstack/oslo.limit/commit/8b3edee1902f5b410da1f5ed872c91afb3a7fc60
Submitter: "Zuul (22348)"
Branch: master

commit 8b3edee1902f5b410da1f5ed872c91afb3a7fc60
Author: Takashi Kajinami <email address hidden>
Date: Tue Sep 16 11:27:41 2025 +0900

    Fix endpoint query

    oepnstacksdk returns generators, not lists, so we have to convert
    the returned values before length check or index access.

    Also keystone does not accept endpoint_type with URL suffix.

    Closes-Bug: #2123895
    Change-Id: Icbf1eb9bab08690a35e402e7a5bd1e0b8912e88d
    Signed-off-by: Takashi Kajinami <email address hidden>

Changed in oslo.limit:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.limit (stable/2025.2)

Fix proposed to branch: stable/2025.2
Review: https://review.opendev.org/c/openstack/oslo.limit/+/961384

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.limit (stable/2025.1)

Fix proposed to branch: stable/2025.1
Review: https://review.opendev.org/c/openstack/oslo.limit/+/961387

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.limit (stable/2025.1)

Reviewed: https://review.opendev.org/c/openstack/oslo.limit/+/961387
Committed: https://opendev.org/openstack/oslo.limit/commit/30a358493c5bf4c523c9e6923cfc1f0fa2743b26
Submitter: "Zuul (22348)"
Branch: stable/2025.1

commit 30a358493c5bf4c523c9e6923cfc1f0fa2743b26
Author: Takashi Kajinami <email address hidden>
Date: Tue Sep 16 11:27:41 2025 +0900

    Fix endpoint query

    oepnstacksdk returns generators, not lists, so we have to convert
    the returned values before length check or index access.

    Also keystone does not accept endpoint_type with URL suffix.

    Closes-Bug: #2123895
    Change-Id: Icbf1eb9bab08690a35e402e7a5bd1e0b8912e88d
    Signed-off-by: Takashi Kajinami <email address hidden>
    (cherry picked from commit 8b3edee1902f5b410da1f5ed872c91afb3a7fc60)
    (cherry picked from commit b14c31dbaac99edda8c60a4f5fca49e0cce5acbb)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.limit (stable/2025.2)

Reviewed: https://review.opendev.org/c/openstack/oslo.limit/+/961384
Committed: https://opendev.org/openstack/oslo.limit/commit/b14c31dbaac99edda8c60a4f5fca49e0cce5acbb
Submitter: "Zuul (22348)"
Branch: stable/2025.2

commit b14c31dbaac99edda8c60a4f5fca49e0cce5acbb
Author: Takashi Kajinami <email address hidden>
Date: Tue Sep 16 11:27:41 2025 +0900

    Fix endpoint query

    oepnstacksdk returns generators, not lists, so we have to convert
    the returned values before length check or index access.

    Also keystone does not accept endpoint_type with URL suffix.

    Closes-Bug: #2123895
    Change-Id: Icbf1eb9bab08690a35e402e7a5bd1e0b8912e88d
    Signed-off-by: Takashi Kajinami <email address hidden>
    (cherry picked from commit 8b3edee1902f5b410da1f5ed872c91afb3a7fc60)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.