Fail to use SR-IOV ports when VFIO is loaded

[Ludovic Beliveau]

Issue has been reproduced in kilo.

There is a limitation when using VFIO with some PCI devices. Some PCI devices with multiple hardware components are places in the same IOMMU group (for example a NIC with multiple ports). In this case, the multiple PCI devices in that same IOMMU group can't be assigned to multiple guests, they are limited to be used by only one guest.

It is possible to specify the driver name when generating the libvirt XML for a guest (name can be "vfio" or "kvm" for legacy PCI device assignment method).

The proposed solution is to provide a configuration parameter to specify which driver to use when using PCI passthrough devices. Perhaps this can be specified by hardware model (PCI vendor and product).

The issue was also described here: https://ask.openstack.org/en/question/62749/vfio-pci-vs-pci-assign-for-sr-iov-direct-ports/

[Baodong (Robert) Li (baoli)]

I haven't seen a case where multiple VFs are placed into the same iommu_group. I have a 3.13 kernel, cisco VICs and intel adaptors, though. Not sure if the kernel version matters in iommu_group placement. Curious about what you have on your system in which multiple VFs are placed in the same iommu_group.

libvrit XML allows specification of either 'vfio' or kvm as the hostdev interface driver. So I think it makes sense to provide such support in nova. As described in the bug description, this can be done
    -- through configuration. One way to do this is to specify the driver as part of the whitelist definition. A simpler way is to specify the driver per compute node.
    -- preconfigured table if the driver can be determined by the device model (based on vendor id and product id). This is preferred if it's possible.

[Ludovic Beliveau (ludovic-beliveau)]

Problem is when you use a PCI Passthrough on a NIC port (not SR-IOV).

Perhaps the description needs to be changed.

[Sylvain Bauza (sylvain-bauza)]

So, my take on that is that this is not technically a bug, rather a current limitation (hence Opinion/Wishlist)

That needs at least a blueprint to be created and possibly some discussion around the implementation design, see https://wiki.openstack.org/wiki/Nova/Liberty_Release_Schedule#How_do_I_get_my_code_merged.3F