R2.21.x latest image
This was seen in CI sanity . All VM launches were failing.
Ashok analyzed that vif.py and vrouter-port-control scripts are not compatible in R2.21.x builds.
nova-compute log :
==================
2016-02-09 22:54:57.097 31482 INFO nova.scheduler.client.report [-] Compute_service record updated for ('ci-oc-subslave-ubuntu-10-84-58-11-1', 'ci-oc-subslave-ubuntu-10-84-58-11-1')
2016-02-09 22:54:57.097 31482 INFO nova.compute.resource_tracker [-] Compute_service record updated for ci-oc-subslave-ubuntu-10-84-58-11-1:ci-oc-subslave-ubuntu-10-84-58-11-1
2016-02-09 22:54:57.348 31482 ERROR nova.compute.manager [-] [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] Instance failed to spawn
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] Traceback (most recent call last):
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2264, in _build_resources
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] yield resources
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2134, in _build_and_run_instance
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] block_device_info=block_device_info)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2627, in spawn
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] block_device_info, disk_info=disk_info)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4444, in _create_domain_and_network
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] self.plug_vifs(instance, network_info)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 952, in plug_vifs
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] self.vif_driver.plug(instance, vif)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 650, in plug
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] func(instance, vif)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 628, in plug_vrouter
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] self._vrouter_port_add(instance, vif)
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 596, in _vrouter_port_add
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] cmd_args))
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] VirtualInterfacePlugException: Failed to create the vRouter port with the command: vrouter-port-control --oper=add --uuid=ba4ffa60-f13d-4d3f-bd36-fe7beaad3d98 --instance_uuid=cf2fd87d-91b7-4549-a62d-3882d85f0bcb --vn_uuid=8edf9cd6-75e7-4b90-b734-c8c155f65a15 --vm_project_uuid=a152751162354c1586bcb75f0e96c378 --ip_address=14.1.1.3 --ipv6_address=None --vm_name=vmc1 --mac=02:ba:4f:fa:60:f1 --tap_name=tapba4ffa60-f1 --port_type=NovaVMPort --vif_type=Vrouter --tx_vlan_id=-1 --rx_vlan_id=-1
2016-02-09 22:54:57.348 31482 TRACE nova.compute.manager [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb]
2016-02-09 22:54:57.349 31482 AUDIT nova.compute.manager [req-5b575eb7-4d88-40b8-aa9a-c1be0bdeb58e None] [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] Terminating instance
2016-02-09 22:54:57.390 31482 WARNING nova.virt.libvirt.driver [-] [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] During wait destroy, instance disappeared.
2016-02-09 22:54:57.616 31482 INFO nova.virt.libvirt.driver [-] [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] Deleting instance files /var/lib/nova/instances/cf2fd87d-91b7-4549-a62d-3882d85f0bcb_del
2016-02-09 22:54:57.617 31482 INFO nova.virt.libvirt.driver [-] [instance: cf2fd87d-91b7-4549-a62d-3882d85f0bcb] Deletion of /var/lib/nova/instances/cf2fd87d-91b7-4549-a62d-3882d85f0bcb_del complete
Review in progress for https:/ /review. opencontrail. org/17073
Submitter: Ashok Singh (<email address hidden>)