I looked through the logs briefly. The failure is with the request from nova-compute to libvirt. 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [req-a09e4e55-6e89-47d7-a0d4-b45bc2ba69c5 204c0a61b28f40b3bcbd1805179a8ae5 bb5b3bb7440d4cbc8b279e1f895551e2 - default default] [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] attaching network adapter failed.: libvirtError: internal error: No more available PCI slots 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] Traceback (most recent call last): 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1642, in attach_interface 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] guest.attach_device(cfg, persistent=True, live=live) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 322, in attach_device 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] self._domain.attachDeviceFlags(device_xml, flags=flags) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] result = proxy_call(self._autowrap, f, *args, **kwargs) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] rv = execute(f, *args, **kwargs) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] six.reraise(c, e, tb) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] rv = meth(*args, **kwargs) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 587, in attachDeviceFlags 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2018-09-02 01:01:36.627 57065 ERROR nova.virt.libvirt.driver [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] libvirtError: internal error: No more available PCI slots I see that the detaches were failing in libvirt, but unfortunately nova ignores this and removes it. 2018-09-02 00:58:11.850 57065 WARNING nova.virt.libvirt.driver [req-faaca6c7-48bd-4b42-8907-78c98819d18d 204c0a61b28f40b3bcbd1805179a8ae5 bb5b3bb7440d4cbc8b279e1f895551e2 - default default] [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] Detaching interface fa:16:3e:7a:52:7a failed because the device is no longer found on the guest. 2018-09-02 00:58:12.524 57065 INFO nova.compute.manager [req-29196fd3-9d22-4255-ab8b-d9a16a68e4f8 c3c9ebd4b37e4ee2b3c93c2b25d490e0 fd5046f6d69a442982ae832c6d0a8e15 - default default] [instance: 9ec7cfa8-18f4-4baa-9801-d980f25fb34a] Neutron deleted interface 88f17a31-7e1e-4c6c-a4a0-3340fac00f7c; detaching it from the instance and deleting it from the info cache VM information as seen from virsh: compute-0:/home/wrsroot# virsh qemu-monitor-command --hmp instance-0000003d "info pci" Bus 0, device 0, function 0: Host bridge: PCI device 8086:1237 id "" Bus 0, device 1, function 0: ISA bridge: PCI device 8086:7000 id "" Bus 0, device 1, function 1: IDE controller: PCI device 8086:7010 BAR4: I/O at 0xc0a0 [0xc0af]. id "" Bus 0, device 1, function 2: USB controller: PCI device 8086:7020 IRQ 11. BAR4: I/O at 0xc040 [0xc05f]. id "usb" Bus 0, device 1, function 3: Bridge: PCI device 8086:7113 IRQ 9. id "" Bus 0, device 2, function 0: VGA controller: PCI device 1013:00b8 BAR0: 32 bit prefetchable memory at 0xfc000000 [0xfdffffff]. BAR1: 32 bit memory at 0xfebd0000 [0xfebd0fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe]. id "video0" Bus 0, device 3, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 11. BAR0: I/O at 0xc060 [0xc07f]. BAR1: 32 bit memory at 0xfebd1000 [0xfebd1fff]. BAR4: 64 bit prefetchable memory at 0xfe000000 [0xfe003fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net0" Bus 0, device 4, function 0: SCSI controller: PCI device 1af4:1001 IRQ 11. BAR0: I/O at 0xc000 [0xc03f]. BAR1: 32 bit memory at 0xfebd2000 [0xfebd2fff]. BAR4: 64 bit prefetchable memory at 0xfe004000 [0xfe007fff]. id "virtio-disk0" Bus 0, device 5, function 0: Class 0255: PCI device 1af4:1002 IRQ 10. BAR0: I/O at 0xc080 [0xc09f]. BAR4: 64 bit prefetchable memory at 0xfe008000 [0xfe00bfff]. id "balloon0" Bus 0, device 6, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1000 [0x101f]. BAR1: 32 bit memory at 0x40044000 [0x40044fff]. BAR4: 64 bit prefetchable memory at 0x40040000 [0x40043fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net1" Bus 0, device 7, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1020 [0x103f]. BAR1: 32 bit memory at 0x40045000 [0x40045fff]. BAR4: 64 bit prefetchable memory at 0x40048000 [0x4004bfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net2" Bus 0, device 8, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1040 [0x105f]. BAR1: 32 bit memory at 0x40046000 [0x40046fff]. BAR4: 64 bit prefetchable memory at 0x4004c000 [0x4004ffff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net3" Bus 0, device 9, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1060 [0x107f]. BAR1: 32 bit memory at 0x40047000 [0x40047fff]. BAR4: 64 bit prefetchable memory at 0x40050000 [0x40053fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net4" Bus 0, device 10, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1080 [0x109f]. BAR1: 32 bit memory at 0x40058000 [0x40058fff]. BAR4: 64 bit prefetchable memory at 0x40054000 [0x40057fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net5" Bus 0, device 11, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x10a0 [0x10bf]. BAR1: 32 bit memory at 0x40059000 [0x40059fff]. BAR4: 64 bit prefetchable memory at 0x4005c000 [0x4005ffff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net6" Bus 0, device 12, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x10c0 [0x10df]. BAR1: 32 bit memory at 0x4005a000 [0x4005afff]. BAR4: 64 bit prefetchable memory at 0x40060000 [0x40063fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net7" Bus 0, device 13, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x10e0 [0x10ff]. BAR1: 32 bit memory at 0x4005b000 [0x4005bfff]. BAR4: 64 bit prefetchable memory at 0x40064000 [0x40067fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net8" Bus 0, device 14, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1400 [0x141f]. BAR1: 32 bit memory at 0x4006c000 [0x4006cfff]. BAR4: 64 bit prefetchable memory at 0x40068000 [0x4006bfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net9" Bus 0, device 15, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1420 [0x143f]. BAR1: 32 bit memory at 0x4006d000 [0x4006dfff]. BAR4: 64 bit prefetchable memory at 0x40070000 [0x40073fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net10" Bus 0, device 16, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1440 [0x145f]. BAR1: 32 bit memory at 0x4006e000 [0x4006efff]. BAR4: 64 bit prefetchable memory at 0x40074000 [0x40077fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net11" Bus 0, device 17, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1460 [0x147f]. BAR1: 32 bit memory at 0x4006f000 [0x4006ffff]. BAR4: 64 bit prefetchable memory at 0x40078000 [0x4007bfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net12" Bus 0, device 18, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1480 [0x149f]. BAR1: 32 bit memory at 0x40380000 [0x40380fff]. BAR4: 64 bit prefetchable memory at 0x4007c000 [0x4007ffff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net13" Bus 0, device 19, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x14a0 [0x14bf]. BAR1: 32 bit memory at 0x40381000 [0x40381fff]. BAR4: 64 bit prefetchable memory at 0x40384000 [0x40387fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net14" Bus 0, device 20, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x14c0 [0x14df]. BAR1: 32 bit memory at 0x40382000 [0x40382fff]. BAR4: 64 bit prefetchable memory at 0x40388000 [0x4038bfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net15" Bus 0, device 21, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x14e0 [0x14ff]. BAR1: 32 bit memory at 0x40383000 [0x40383fff]. BAR4: 64 bit prefetchable memory at 0x4038c000 [0x4038ffff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net16" Bus 0, device 22, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1800 [0x181f]. BAR1: 32 bit memory at 0x40394000 [0x40394fff]. BAR4: 64 bit prefetchable memory at 0x40390000 [0x40393fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net17" Bus 0, device 23, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1820 [0x183f]. BAR1: 32 bit memory at 0x40395000 [0x40395fff]. BAR4: 64 bit prefetchable memory at 0x40398000 [0x4039bfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net18" Bus 0, device 24, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1840 [0x185f]. BAR1: 32 bit memory at 0x40396000 [0x40396fff]. BAR4: 64 bit prefetchable memory at 0x4039c000 [0x4039ffff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net19" Bus 0, device 25, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1860 [0x187f]. BAR1: 32 bit memory at 0x40397000 [0x40397fff]. BAR4: 64 bit prefetchable memory at 0x403a0000 [0x403a3fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net20" Bus 0, device 26, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1880 [0x189f]. BAR1: 32 bit memory at 0x403a8000 [0x403a8fff]. BAR4: 64 bit prefetchable memory at 0x403a4000 [0x403a7fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net21" Bus 0, device 27, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x18a0 [0x18bf]. BAR1: 32 bit memory at 0x403a9000 [0x403a9fff]. BAR4: 64 bit prefetchable memory at 0x403ac000 [0x403affff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net22" Bus 0, device 28, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x18c0 [0x18df]. BAR1: 32 bit memory at 0x403aa000 [0x403aafff]. BAR4: 64 bit prefetchable memory at 0x403b0000 [0x403b3fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net23" Bus 0, device 29, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x18e0 [0x18ff]. BAR1: 32 bit memory at 0x403ab000 [0x403abfff]. BAR4: 64 bit prefetchable memory at 0x403b4000 [0x403b7fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net24" Bus 0, device 30, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1c00 [0x1c1f]. BAR1: 32 bit memory at 0x403bc000 [0x403bcfff]. BAR4: 64 bit prefetchable memory at 0x403b8000 [0x403bbfff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net25" Bus 0, device 31, function 0: Ethernet controller: PCI device 1af4:1000 IRQ 0. BAR0: I/O at 0x1c20 [0x1c3f]. BAR1: 32 bit memory at 0x403bd000 [0x403bdfff]. BAR4: 64 bit prefetchable memory at 0x40700000 [0x40703fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "net26" compute-0:/home/wrsroot# virsh dumpxml instance-0000003d instance-0000003d 7a3df365-5a01-41d2-bf6e-a4839195fecb tenant2-if_attach_tenant-2 2018-09-07 19:35:37 1024 2 0 0 1 tenant2 tenant2 1048576 1048576 1 /machine Fedora Project OpenStack Nova 16.0.2-1.tis.5 669bedc8-48c1-47de-ba20-d7f323614df5 7a3df365-5a01-41d2-bf6e-a4839195fecb Virtual Machine hvm destroy restart destroy /usr/libexec/qemu-kvm 17851e43-e880-4e2b-aea5-27eb3dd0d7c8