sr-iov: please add support for setting supported_pci_vendor_devs under the ml2_sriov section

[JuanJo Ciarlante]

Stock packages come with a default
supported_pci_vendor_devs=15b3:1004,8086:10ca
(mitaka pkgs), which can obviously get easily outdated.

To make sr-iov work on ixgbevf hardware I had to workaround it by
adding below two lines to neutron-api charm templates/kilo/ml2_conf.ini:

[ml2_sriov]
supported_pci_vendor_devs = 15b3:1004,8086:10ca,8086:10ed

[JuanJo Ciarlante (jjo)]

FYI for mellanox cards w/mlx5_core, lspci -nn:

81:01.2 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016]

, I'm using manually added:

[ml2_sriov]
supported_pci_vendor_devs = 15b3:1004,8086:10ca,8086:10ed,15b3:1016

[Nobuto Murata (nobuto)]

I faced this issue when trying to enable SR-IOV with the following device:
04:00.0 Ethernet controller [0200]: Emulex Corporation OneConnect 10Gb NIC (be3) [19a2:0710] (rev 01)

[Frode Nordahl (fnordahl)]

Just chiming in here, this is indeed a requirement for actual usability of the charm to configure SR-IOV functionality

Hardware:
04:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)
04:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528] (rev 01)

Log of failed usage AS-IS:
2017-05-31 11:09:51.015 70030 DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17f2509f-e688-4052-aee2-a9f2965a44b9 158ef64791b2439784e9cfa6077a382f 148d0a576d394bf6947d1d3578b87eda - - -] Attempting to bind port 86afc89d-78ac-464b-bbc3-9b48e84da140 on network 90b95619-a4d4-4ef5-9134-50ae9c3139c5 bind_port /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_agent.py:60
2017-05-31 11:09:51.015 70030 DEBUG neutron.plugins.ml2.drivers.mech_agent [req-17f2509f-e688-4052-aee2-a9f2965a44b9 158ef64791b2439784e9cfa6077a382f 148d0a576d394bf6947d1d3578b87eda - - -] Refusing to bind due to unsupported vnic_type: direct bind_port /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_agent.py:65
2017-05-31 11:09:51.016 70030 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-17f2509f-e688-4052-aee2-a9f2965a44b9 158ef64791b2439784e9cfa6077a382f 148d0a576d394bf6947d1d3578b87eda - - -] Attempting to bind port 86afc89d-78ac-464b-bbc3-9b48e84da140 on network 90b95619-a4d4-4ef5-9134-50ae9c3139c5 bind_port /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:98
2017-05-31 11:09:51.016 70030 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-17f2509f-e688-4052-aee2-a9f2965a44b9 158ef64791b2439784e9cfa6077a382f 148d0a576d394bf6947d1d3578b87eda - - -] Unsupported pci_vendor 8086:1515 _check_supported_pci_vendor_device /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:179
2017-05-31 11:09:51.017 70030 DEBUG neutron.plugins.ml2.drivers.mech_sriov.mech_driver.mech_driver [req-17f2509f-e688-4052-aee2-a9f2965a44b9 158ef64791b2439784e9cfa6077a382f 148d0a576d394bf6947d1d3578b87eda - - -] Refusing to bind due to unsupported pci_vendor device bind_port /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py:110

Manual resolution / workaround:
- Add 'supported_pci_vendor_devs = 8086:1515' to /etc/neutron/plugins/ml2/ml2_conf_sriov.ini on neutron-api units
- Extend NEUTRON_PLUGIN_CONFIG in /etc/default/neutron-server with: ' --config-file=/etc/neutron/plugins/ml2/ml2_conf_sriov.ini' on neutron-api units

[Frode Nordahl (fnordahl)]

On trusty we would need to update the init configuration: /etc/init/neutron-server.conf
Replace ‘[ -r "$NEUTRON_PLUGIN_CONFIG" ]’ (ca line 21)
With ‘[ -n "$NEUTRON_PLUGIN_CONFIG" ]’

Since newer versions have (at least xenial has) switched from -r to -n, maybe it would be acceptable to SRU such a change to the deb?

[James Page (james-page)]

We should just add things to the standard ml2 configuration file; having the extra file just adds complexity in my opinion.

[ml2_sriov]
supported_pci_vendor_devs = 15b3:1004,8086:10ca,8086:10ed

[Frode Nordahl (fnordahl)]

That would work too I guess, I'll verify.

My only issues with it is that:
a) We would miss any upstream defaults that would be added to the ml2_conf_sriov.ini file over time.
b) We would deviate from what our users would expect wrt. what they find in documentation and forum posts about the topic on the Internet

[Frode Nordahl (fnordahl)]

As a compromise I'll add the ml2_conf_sriov.ini as a template to my proposal with comment referencing ml2_conf.ini.

[Frode Nordahl (fnordahl)]

Fix proposed to branch: master
Review: https://review.openstack.org/#/c/472640/

[Frode Nordahl (fnordahl)]

From Newton an onward this should no longer be necessary. As an alternative we could look into backporting this:
https://bugs.launchpad.net/neutron/+bug/1611302

If it is easy backportable to Mitaka and Liberty, that might be something to consider carrying in our packages instead of adding this complexity to the charms?

I'll verify that it works without this option in Newton and look into the backportability

[Frode Nordahl (fnordahl)]

I have verified that this option indeed is no longer needed from Newton and onward.

I have verified that the above mentioned upstream patch is easily backportable to Mitaka and Liberty, and that the resulting debs show the expected behavior.

The reasoning behind deprecating and no longer having a default value for this option is that Nova with its PCI whitelist already has enough checks. This would also be true for Mitaka and Liberty.

If a end user system currently has a value configured for supported_pci_vendor_devs a updated package will induce no change.

If a end user system currently has _NO_ value configured for supported_pci_vendor_devs (the default), a patched package will make Neutron no longer question or override Nova's scheduling decision in the port binding process.

@James I would ask your input on this; would you entertain adaption of the charms to handle the differences in configuration SR-IOV on OpenStack versions prior to Newton which might require introduction of a new configuration option (see proposed patch in #8), or would you entertain carrying a backport of the patch mentioned in comment #9 in neutron packages for xenial and trusty/liberty UCA.

[Frode Nordahl (fnordahl)]

FTR, I have also verified that the patch in #9 would be easily back-portable to Kilo (the release SR-IOV support was introduced), and that the resulting debs would work as expected.

[James Page (james-page)]

I think we should *not* backport the fix to deprecate the configuration option in older openstack releases; I'm never a huge fan in a behavioural change as part of an SRU (and I suspect neither will the SRU team be).

So I think we'll have to add a configuration option to the charm that only applies in < Newton (with clear annotation of that in the config.yaml/README).

[Frode Nordahl (fnordahl)]

Ack, the proposed solution for the charm in comment #9 is now polished, verified and put through CI, Ready for review.

[Frode Nordahl (fnordahl)]

Referenced the wrong comment in #13, the link to the review is of course in comment #8.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/472640
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-api/commit/?id=353d4a97eb82f67098b1749303cc117aa9c2b6e7
Submitter: Jenkins
Branch: master

commit 353d4a97eb82f67098b1749303cc117aa9c2b6e7
Author: Frode Nordahl <email address hidden>
Date: Fri Jun 9 10:08:30 2017 +0200

    Add support for setting sriov_supported_pci_vendor_devs

    This is required for SR-IOV to work on OpenStack versions
    Kilo, Liberty and Mitaka (unless you are the lucky owner of
    NICs with the default vendor/product IDs '15b3:1004', '8086:10ca').

    The option is deprecated in Newton and the default behaviour
    onwards is to not perform the check (unless configured) and not
    overrule Nova's scheduling decision. (See LP #1611302)

    Re-work mechanism_driver templating. Current implementation
    treats mechanism drivers 'hyperv' and 'sriovnicswitch' as
    mutually exclusive for simplicity. This prohibits us from
    adding functional test for verifying sriov statements in
    configuration files.

    Due to how neutron init scripts are laid out on various Linux
    distributions put the [ml2_sriov] section in ml2_conf.ini instead
    of its default ml2_conf_sriov.ini location.

    Add a placeholder ml2_conf_sriov.ini with comment to point users
    in the right direction.

    Change-Id: I37da1c430a06417ff7f1bc9df2d984137688bba0
    Closes-Bug: #1630387