vhostuser_sock_path not present in vif_details

Bug #1684219 reported by Arslan Qadeer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-ovn
Confirmed
Undecided
Arslan Qadeer

Bug Description

Trying OVN+DPDK
Following is setup configuration:

[ovn]
ovn_nb_connection = tcp:172.31.100.200:6641
ovn_sb_connection = tcp:172.31.100.200:6642
neutron_sync_mode = log
ovn_l3_mode = False
ovn_native_dhcp = True
vhost_sock_dir = /var/run/openvswitch/
vif_type = vhostuser

When VM is launched following error appears in nova logs:

sired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] _get_desirable_cpu_topologies /usr/lib/python2.7/site-packages/nova/virt/hardware.py:602
2017-04-19 11:44:17.945 19233 DEBUG nova.virt.libvirt.vif [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] vif_type=vhostuser instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=True,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2017-04-19T15:44:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='vm1',display_name='vm1',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(3),host='compute1',hostname='vm1',id=96,image_ref='2a33c5ed-2478-4d27-a842-11960cdefbb0',info_cache=InstanceInfoCache,instance_type_id=3,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='compute1',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='compute1',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='8048262017014b7798216d536fe87d3b',ramdisk_id='',reservation_id='r-ulifwklp',root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='2a33c5ed-2478-4d27-a842-11960cdefbb0',image_container_format='bare',image_disk_format='qcow2',image_min_disk='1',image_min_ram='0',network_allocated='True'},tags=<?>,task_state='spawning',terminated_at=None,updated_at=2017-04-19T15:44:16Z,user_data=None,user_id='162aa06ebf244cbd870e8a41e2687378',uuid=dabb91f8-baa2-4453-ae28-118b9be3c874,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"profile": {}, "ovs_interfaceid": "dbdf152c-a72c-47c7-8580-315052ba9cf6", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "1.1.1.5"}], "version": 4, "meta": {}, "dns": [], "routes": [], "cidr": "1.1.1.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "1.1.1.1"}}], "meta": {"injected": false, "tenant_id": "8048262017014b7798216d536fe87d3b", "mtu": 1442}, "id": "cfa3bba9-c5a5-4b91-859f-900f4d81f034", "label": "net_test"}, "devname": "tapdbdf152c-a7", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_mode": "client"}, "address": "fa:16:3e:12:4f:69", "active": false, "type": "vhostuser", "id": "dbdf152c-a72c-47c7-8580-315052ba9cf6", "qbg_params": null} virt_type=kvm get_config /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:511
2017-04-19 11:44:17.946 19233 DEBUG nova.network.os_vif_util [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] Converting VIF {"profile": {}, "ovs_interfaceid": "dbdf152c-a72c-47c7-8580-315052ba9cf6", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "1.1.1.5"}], "version": 4, "meta": {}, "dns": [], "routes": [], "cidr": "1.1.1.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "1.1.1.1"}}], "meta": {"injected": false, "tenant_id": "8048262017014b7798216d536fe87d3b", "mtu": 1442}, "id": "cfa3bba9-c5a5-4b91-859f-900f4d81f034", "label": "net_test"}, "devname": "tapdbdf152c-a7", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_mode": "client"}, "address": "fa:16:3e:12:4f:69", "active": false, "type": "vhostuser", "id": "dbdf152c-a72c-47c7-8580-315052ba9cf6", "qbg_params": null} nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:362
2017-04-19 11:44:17.946 19233 DEBUG nova.network.os_vif_util [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] No conversion for VIF type vhostuser yet nova_to_osvif_vif /usr/lib/python2.7/site-packages/nova/network/os_vif_util.py:378
2017-04-19 11:44:17.947 19233 WARNING nova.virt.osinfo [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] Cannot find OS information - Reason: (No configuration information found for operating system Empty)
2017-04-19 11:44:17.947 19233 DEBUG nova.virt.libvirt.vif [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] OKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK _get_vhostuser_settings /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:397
2017-04-19 11:44:17.947 19233 DEBUG nova.virt.libvirt.vif [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] vhostuser_socket _get_vhostuser_settings /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:398
2017-04-19 11:44:17.947 19233 DEBUG nova.virt.libvirt.vif [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] OKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK _get_vhostuser_settings /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:402
2017-04-19 11:44:17.947 19233 DEBUG nova.virt.libvirt.vif [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] OKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK _get_vhostuser_settings /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:404
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] Instance failed to spawn
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] Traceback (most recent call last):
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2078, in _build_resources
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] yield resources
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1920, in _build_and_run_instance
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] block_device_info=block_device_info)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2611, in spawn
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] block_device_info=block_device_info)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4658, in _get_guest_xml
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] context)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4490, in _get_guest_config
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] flavor, virt_type, self._host)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 531, in get_config
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] inst_type, virt_type, host)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 416, in get_config_vhostuser
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] mode, sock_path = self._get_vhostuser_settings(vif)
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 409, in _get_vhostuser_settings
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] vif_id=vif['id'])
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] VifDetailsMissingVhostuserSockPath: vhostuser_sock_path not present in vif_details for vif dbdf152c-a72c-47c7-8580-315052ba9cf6
2017-04-19 11:44:17.948 19233 ERROR nova.compute.manager [instance: dabb91f8-baa2-4453-ae28-118b9be3c874]
2017-04-19 11:44:17.948 19233 INFO nova.compute.manager [req-a34f2081-66a5-43c7-aa6d-8a343f48b284 162aa06ebf244cbd870e8a41e2687378 8048262017014b7798216d536fe87d3b - - -] [instance: dabb91f8-baa2-4453-ae28-118b9be3c874] Terminating instance

From the above logs it is clear that nova cannot get socket path, whereas in case of OVS neutron-openvswitch-agent perfectly posts socket path as shown below:

2017-04-19 11:06:26.886 18916 DEBUG nova.virt.libvirt.vif [req-7833f1ed-9fb6-4193-b76d-ddbb0a99cfb7 433310d31f4b4b949368325aa7d53216 7d589ab4b1334533a02f6030c1c2e2e2 - - -] vif_type=vhostuser instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=True,availability_zone='nova',cell_name=None,cleaned=False,config_drive='',created_at=2017-04-19T15:06:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='vm1',display_name='vm1',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(3),host='aio',hostname='vm1',id=6,image_ref='4a1f4c2f-e858-4718-b72c-f0d8aaf98ed6',info_cache=InstanceInfoCache,instance_type_id=3,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='aio',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='aio',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7d589ab4b1334533a02f6030c1c2e2e2',ramdisk_id='',reservation_id='r-jmqzafo3',root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={image_base_image_ref='4a1f4c2f-e858-4718-b72c-f0d8aaf98ed6',image_container_format='bare',image_disk_format='qcow2',image_min_disk='1',image_min_ram='0',network_allocated='True'},tags=<?>,task_state='spawning',terminated_at=None,updated_at=2017-04-19T15:06:25Z,user_data=None,user_id='433310d31f4b4b949368325aa7d53216',uuid=46f77ffc-5d9e-4ca8-9339-bf79af11a721,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"profile": {}, "ovs_interfaceid": "34df6be1-f955-43a4-9d5f-6c3b903cba17", "preserve_on_delete": false, "network": {"bridge": "br-int", "subnets": [{"ips": [{"meta": {}, "version": 4, "type": "fixed", "floating_ips": [], "address": "1.1.1.5"}], "version": 4, "meta": {"dhcp_server": "1.1.1.2"}, "dns": [], "routes": [], "cidr": "1.1.1.0/24", "gateway": {"meta": {}, "version": 4, "type": "gateway", "address": "1.1.1.1"}}], "meta": {"injected": false, "tenant_id": "7d589ab4b1334533a02f6030c1c2e2e2", "mtu": 1450}, "id": "befcfc66-1aa2-4d5f-8ccb-b3c97d11eb9f", "label": "net_demo"}, "devname": "tap34df6be1-f9", "vnic_type": "normal", "qbh_params": null, "meta": {}, "details": {"port_filter": false, "vhostuser_ovs_plug": true, "vhostuser_socket": "/var/run/openvswitch/vhu34df6be1-f9", "vhostuser_mode": "client"}, "address": "fa:16:3e:e3:31:e8", "active": false, "type": "vhostuser", "id": "34df6be1-f955-43a4-9d5f-6c3b903cba17", "qbg_params": null} virt_type=kvm get_config /usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py:499

Steps to reproduce:
Install DPDK enabled OVS and OVN packages configure ml2_config.ini as mentioned above and try to launch a VM

Is there any workaround for that?

Revision history for this message
Arslan Qadeer (arslanqxgrid) wrote :

It is working on networking_ovn (Master) with OpenStack (stable/ocata).

Could anyone please backport this to Newton as well and give details what was changed on DPDK side from Newton TO Master.

Changed in networking-ovn:
status: New → Incomplete
status: Incomplete → Opinion
status: Opinion → New
Changed in networking-ovn:
status: New → Confirmed
status: Confirmed → Incomplete
status: Incomplete → Confirmed
assignee: nobody → Arslan Qadeer (arslanqxgrid)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-ovn (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/468990

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-ovn (stable/newton)

Change abandoned by Tony Breeds (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/468990
Reason: This branch (stable/newton) is at End Of Life

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.