SRIOV PF/VF allocation fails with NUMA aware flavor

Bug #1590746 reported by Ricardo Noriega on 2016-06-09
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Vladik Romanovsky

Bug Description

It seems that the main failure happens due to the incorrect NUMA filtering in the pci allocation mechanism. The allocation is being done according to the instance NUMA topology, however, this is not always correct. Specifically in the case when a user selects hw:numa_nodes=1, which would mean that VM will take resources from just one numa node and not from a specific one.

Steps to reproduce

Create nova flavor with NUMA awareness, CPU pinning, Huge pages, etc:

# nova flavor-create prefer_pin_1 auto 2048 20 1
# nova flavor-key prefer_pin_1 set hw:numa_nodes=1
# nova flavor-key prefer_pin_1 set hw:mem_page_size=1048576
# nova flavor-key prefer_pin_1 set hw:numa_mempolicy=strict
# nova flavor-key prefer_pin_1 set hw:cpu_policy=dedicated
# nova flavor-key prefer_pin_1 set hw:cpu_thread_policy=prefer

Then instantiate VMs with direct-physical neutron ports:

neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf1
nova boot pf1 --flavor prefer_pin_1 --image centos_udev --nic port-id=a0fe88f6-07cc-4c70-b702-1915e36ed728
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf2
nova boot pf2 --flavor prefer_pin_1 --image centos_udev --nic port-id=b96de3ec-ef94-428b-96bc-dc46623a2427

Third VM instantiation failed. Our environment has got 4 NICs configured to be allocated. However, with a regular flavor (m1.normal), the instantiation works:

neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf3
nova boot pf3 --flavor 2 --image centos_udev --nic port-id=52caacfe-0324-42bd-84ad-9a54d80e8fbe
neutron port-create nfv_sriov --binding:vnic-type direct-physical --name pf4
nova boot pf4 --flavor 2 --image centos_udev --nic port-id=7335a9a6-82d0-4595-bb88-754678db56ef

Expected result

PCI passthrough (PFs and VFs) should work in an environment with NUMATopologyFilter enable

Actual result

Checking availability of NICs with NUMATopologyFilter is not working.


1 controller + 1 compute.

OpenStack Mitaka

Logs & Configs

See attachment

Ricardo Noriega (rnoriega-b) wrote :
Changed in nova:
assignee: nobody → Vladik Romanovsky (vladik-romanovsky)
tags: added: numa scheduler
Changed in nova:
status: New → In Progress

Change abandoned by Vladik Romanovsky (<email address hidden>) on branch: master

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers