During instance creation, it fails due to port in use exception.
Nova logs :
2016-05-27 02:47:03.039 2621 AUDIT nova.compute.claims [req-d513a6b2-38a4-43fe-97d5-e8f56dbab9f5 c6032e50916e4125b179f20216393e8d 0d9f2e7e39a5472c848fe694dd9ad5fd] [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] Claim successful
2016-05-27 02:47:03.055 2621 INFO nova.virt.libvirt.driver [req-fceb143d-304d-431f-9406-ff731dd205c0 c6032e50916e4125b179f20216393e8d 0d9f2e7e39a5472c848fe694dd9ad5fd] [instance: 5abe6c70-a0f5-4613-a77f-69d17db83b76] Creating image
2016-05-27 02:47:03.502 2621 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager Traceback (most recent call last):
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1530, in _allocate_network_async
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager dhcp_options=dhcp_options)
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 241, in allocate_for_instance
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager raise exception.PortInUse(port_id=port_id)
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager PortInUse: Port ee46d923-735d-4bf8-86dc-4c38c60a3a8f is still in use.
2016-05-27 02:47:03.502 2621 TRACE nova.compute.manager
2016-05-27 02:47:04.100 2621 INFO nova.compute.manager [-] [instance: 3db02432-eab7-4459-bfe2-4b5f3b18d777] VM Started (Lifecycle Event)
2016-05-27 02:47:04.106 2621 INFO nova.virt.libvirt.driver [-] [instance: 3db02432-eab7-4459-bfe2-4b5f3b18d777] Instance spawned successfully.
2016-05-27 02:47:04.113 2621 INFO nova.virt.libvirt.driver [req-d513a6b2-38a4-43fe-97d5-e8f56dbab9f5 c6032e50916e4125b179f20216393e8d 0d9f2e7e39a5472c848fe694dd9ad5fd] [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] Creating image
2016-05-27 02:47:04.212 2621 INFO nova.compute.manager [-] [instance: 3db02432-eab7-4459-bfe2-4b5f3b18d777] During sync_power_state the instance has a pending task (spawning). Skip.
2016-05-27 02:47:04.885 2621 ERROR nova.compute.manager [req-d513a6b2-38a4-43fe-97d5-e8f56dbab9f5 c6032e50916e4125b179f20216393e8d 0d9f2e7e39a5472c848fe694dd9ad5fd] [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] Instance failed to spawn
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] Traceback (most recent call last):
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1740, in _spawn
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] block_device_info)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2290, in spawn
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] admin_pass=admin_password)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2760, in _create_image
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] instance, network_info, admin_pass, files, suffix)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2559, in _inject_data
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] net = netutils.get_injected_network_template(network_info)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/virt/netutils.py", line 71, in get_injected_network_template
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] if not (network_info and template):
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 424, in __len__
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] return self._sync_wrapper(fn, *args, **kwargs)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 411, in _sync_wrapper
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] self.wait()
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 449, in wait
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] self[:] = self._gt.wait()
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] return self._exit_event.wait()
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] current.throw(*self._exc)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] result = function(*args, **kwargs)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1530, in _allocate_network_async
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] dhcp_options=dhcp_options)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 241, in allocate_for_instance
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] raise exception.PortInUse(port_id=port_id)
2016-05-27 02:47:04.885 2621 TRACE nova.compute.manager [instance: 897125c2-3176-415f-8f2d-0ebf48ecc53d] PortInUse: Port ee46d923-735d-4bf8-86dc-4c38c60a3a8f is still in use.
Contrail-version : 2.21-102
1:2014.1.3-0ubuntu1~cloud0.3contrail
Host OS : Ubuntu Trusty
From the above logs, it seems that the port ee46d923- 735d-4bf8- 86dc-4c38c60a3a 8f is already attached to another VM. What is the history of this port? Please provide exact sequence of commands that resulted in this failure.