Cannot assign 2 same type PCI passthrough devices to 2 instances separately

[Yi Liu]

I want configured the PCI pass-through to assign the physical NIC to instance.

I have two NICs with same type:
04:00.0 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFP+ Network Connection [8086:154d] (rev 01)
04:00.1 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFP+ Network Connection [8086:154d] (rev 01)

I add below configuration into nova.conf on compute node:
pci_passthrough_whitelist=[{"vendor_id":"8086", "product_id":"154d"}]

and add below configuration into nova.conf on controller node:
pci_alias={"vendor_id":"8086", "product_id":"154d", "name":"a1"}

then create flavor and set the extra spec:
nova flavor-key test_flavor set "pci_passthrough:alias"="a1:1"

After that, I launch the first instance using the test_flavor, it succeed.

However, I launch the second instance also using the test_flavor, but it failed.

There is error log from scheduler.log, it said that the device is already in use.
I found that when launch the second instance, nova still assign the first NIC which has been already assigned to the first instance to the second instance. So it failed...

I suppose this maybe a bug of nova, or could be resolved by configuration. ---Anyway, need your confirmation!

Below is the error log for reference:
2013-12-05 05:21:10.522 6109 ERROR nova.scheduler.filter_scheduler [req-6abe0403-36fe-4d24-9852-9436fbf75331 c7095324d1fe439e8c9f350a71690388 92a596418bac4d3a901f1956fbed4463] [instance: 8dcb7465-080b-4bb6-bcd6-d228ca17e5fd] Error from last host: lng-compute-1 (node lng-compute-1): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1037, in _build_instance\n set_access_ip=set_access_ip)\n', u' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1410, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', u' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1407, in _spawn\n block_device_info)\n', u' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2070, in spawn\n block_device_info, context=context)\n', u' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3205, in _create_domain_and_network\n domain = self._create_domain(xml, instance=instance, power_on=power_on)\n', u' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3148, in _create_domain\n domain.XMLDesc(0))\n', u' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3143, in _create_domain\n domain.createWithFlags(launch_flags)\n', u' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', u' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in proxy_call\n rv = execute(f,*args,**kwargs)\n', u' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker\n rv = meth(*args,**kwargs)\n', u' File "/usr/lib64/python2.6/site-packages/libvirt.py", line 708, in createWithFlags\n if ret == -1: raise libvirtError (\'virDomainCreateWithFlags() failed\', dom=self)\n', u'libvirtError: Requested operation is not valid: PCI device 0000:04:00.0 is in use by domain instance-00000010\n']

[Shuangtai Tian (shuangtai-tian)]

Hi, Yi liu
can you give more details, for example : if the stats in DB is " allocated" ?
you can use the api patch: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/pci-api-support,n,z

[Yi Liu (ryan-yi-liu)]

Hi Shuangtai,
Thank you for your reply.
However, I have reinstalled my environment, then the issue didn't reproduced...
If I met the problem again, I will provide you more information.

[John Garbutt (johngarbutt)]

sounds like we need some digging to work out what is going on here, incomplete

[Sean Dague (sdague)]

currently set to non reproducable, closing for now. Please reopen if this problem happens again.