placement response timeout when create a vm with passthrough of 7GPUs

Bug #2070257 reported by zack chen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
placement
New
Undecided
Unassigned

Bug Description

When attempting to create a virtual machine with direct passthrough of 7 GPUs, nova-scheduler reports a placement response timeout. The placement logs do not indicate any errors but continuously print logs,
I enabled debug logging and executed a curl command that fetched over 200MB of log data.

The curl command is:

curl -g -i --insecure -X GET "https://192.168.1.100:8780//allocation_candidates?group_policy=none&limit=1000&member_of=in%3Aa58d4d6e-dd6f-44a0-9d84-c0da4edd28d9&resources=MEMORY_MB%3A32768%2CVCPU%3A16&resourcesdevice_profile_0=PGPU%3A1&resourcesdevice_profile_1=PGPU%3A1&resourcesdevice_profile_2=PGPU%3A1&resourcesdevice_profile_3=PGPU%3A1&resourcesdevice_profile_4=PGPU%3A1&resourcesdevice_profile_5=PGPU%3A1&resourcesdevice_profile_6=PGPU%3A1&root_required=COMPUTE_ACCELERATORS%2C%21COMPUTE_STATUS_DISABLED" -H "Accept: application/json" -H "OpenStack-API-Version: placement 1.36" -H "User-Agent: openstacksdk/0.101.0 keystoneauth1/5.0.0 python-requests/2.28.2 CPython/3.10.12" -H "X-Auth-Token: token"

The key log entry in placement-api.log is:

placement-api.log:498072:2024-06-22 20:26:19.825 738 DEBUG placement.objects.allocation_candidate [None req-08ad26e6-569d-4edd-bd70-56b579c17d86 1fc2e622be4d4d10ba4d9c9e1665abde 427edb8e80594a39ba95f18af64e6c42 - - default default] Merging candidates yields 20160 allocation requests and 10 provider summaries _merge_candidates /var/lib/kolla/venv/lib/python3.10/site-packages/placement/objects/allocation_candidate.py:809

Does this mean that looping 20160 times caused the timeout?

zack chen (zackchen)
description: updated
summary: - placement response timeout when create a with passthrough of 7GPUs
+ placement response timeout when create a vm with passthrough of 7GPUs
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.