Cant revert from neutron-port PF to VF because sriov_numvfs parameter get "0" value

[Eran Kuris]

Description of problem:

When manage SR-IOV PFs as Neutron ports I can see that
/sys/class/net/enp5s0f1/device/sriov_numvfs parameter gets "0" value.
When I delete the PF port so I can switch to SRIOV - direct port (VF) I can't boot vm because sriov_numvfs parameter equal to "0" value

Version-Release number of selected component (if applicable):

$ rpm -qa |grep neutron
python-neutron-lib-0.3.0-0.20160803002107.405f896.el7ost.noarch
openstack-neutron-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
puppet-neutron-9.1.0-0.20160813031056.7cf5e07.el7ost.noarch
python-neutron-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
openstack-neutron-lbaas-9.0.0-0.20160816191643.4e7301e.el7ost.noarch
python-neutron-fwaas-9.0.0-0.20160817171450.e1ac68f.el7ost.noarch
python-neutron-lbaas-9.0.0-0.20160816191643.4e7301e.el7ost.noarch
openstack-neutron-ml2-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
openstack-neutron-metering-agent-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
openstack-neutron-openvswitch-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
python-neutronclient-5.0.0-0.20160812094704.ec20f7f.el7ost.noarch
openstack-neutron-common-9.0.0-0.20160817153328.b9169e3.el7ost.noarch
openstack-neutron-fwaas-9.0.0-0.20160817171450.e1ac68f.el7ost.noarch

$ rpm -qa |grep nova
python-novaclient-5.0.1-0.20160724130722.6b11a1c.el7ost.noarch
openstack-nova-api-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
puppet-nova-9.1.0-0.20160813014843.b94f0a0.el7ost.noarch
openstack-nova-common-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
openstack-nova-novncproxy-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
openstack-nova-conductor-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
python-nova-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
openstack-nova-scheduler-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
openstack-nova-cert-14.0.0-0.20160817225441.04cef3b.el7ost.noarch
openstack-nova-console-14.0.0-0.20160817225441.04cef3b.el7ost.noarch

How reproducible:

Always

Steps to Reproduce:

1.Set SRIOV ENV and PF support : https://docs.google.com/document/d/1qQbJlLI1hSlE4uwKpmVd0BoGSDBd8Z0lTzx5itQ6WL0/edit#
2. BOOT VM that assign to PF (neutron port- direct-physical) - should boot well
3. check cat /sys/class/net/enp5s0f1/device/sriov_numvfs (= 0)
4. delete vm and check again sriov_numvfs (=0)
5. I expect that numvfs should return to the default value that was configured

[Stephen Finucane (stephenfinucane)]

You need to create these ports yourself: nova/neutron will use the ports, but will not create them on the host. Please refer to the SR-IOV documentation [1] for information on how to do this.

[1] http://docs.openstack.org/mitaka/networking-guide/config-sriov.html#create-virtual-functions-compute

[Eran Kuris (ekuris)]

I think it valid case that user will revert from PF to VF port.
When we create vm with PF port (direct physical ) /sys/class/net/enp5s0f1/device/sriov_numvfs parameter gets "0" value which is expected , but when I delete this port I expected that /sys/class/net/enp5s0f1/device/sriov_numvfs will return to previous value so I can set VF ports.
the problem still exist on OSP-1o - newton

[Andrey Volkov (avolkov)]

@ekuris Do you now who is responsible for update value in sriov_numvfs? I didn't find any code in nova related to this. I suppose the issue is related to neutron or even some configuration process as stephenfinucane mentioned.

@smatov Could you comment on that issue?

[Sergey Matov (smatov)]

Hello.

I might be wrong but probably the reason of this issue is neutron direct-physical assignation for PF. QEMU will have direct PCI control of PF and host driver will loose control of the nic. So I guess it's side effect caused by system, not OpenStack components.