[RFE] Add new VNIC types for "Cyborg" handled ports

[Xinran WANG]

According to the nova-spec[1], we have agreed to add a new vnic type for port that is associated with a smartnic managed by cyborg. This new vnic type will have similar functionality to "direct", and Nova needs it to check if it should retrieve the "device_profile" in neutron port and bind such devices.

[1]https://specs.openstack.org/openstack/nova-specs/specs/wallaby/approved/support-sriov-smartnic.html#neutron

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Xinran:

Where this VNIC type is going to be used? The network backends support a list of VNIC types, of course each one with some specific particularities.

As an example:
* OVS: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/mech_driver/mech_openvswitch.py#L76-L78
* SR-IOV: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py#L61-L63
* OVN: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py#L160-L163

Can you specify the network backends that are going to use this new VNIC type and how?

Regards.

[Xinran WANG (xinran)]

Hi Rodolfo,

This vnic type has similar functionality as the "direct" type. Nova needs to check the vnic type, if the vnic type is "cyborg", Nova will know this port has a device request storing in the device profile, and then it will trigger the process to retrieve the device profile from the port and do scheduling and so on.

Currently, the SRIOV backends should support this new type. It is sufficient as the first step to support smartnic.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi:

We already have a "smart-nic" VNIC type defined in neutron-lib (currently only used in OVS): https://review.opendev.org/c/openstack/neutron-lib/+/624660 [1]

I guess the purpose of this new VNIC type is different and is currently supported only in SR-IOV.

Regards.

[1]https://storyboard.openstack.org/#!/story/2003346

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Added topic in the agenda for the next Neutron drivers meeting: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda

Next Friday Jan 8.

[Akihiro Motoki (amotoki)]

VNIC type is mainly used to determine how neutron should handle a port (e.g. which mechanism driver should handle the port). Reading the RFE bug description, it not clear on this point. The only need described is to make nova determine whetherr a corresponding port is managed by cybord. Could you clarify what kind of change is required from the point of view of processing a port in neutron?

Note that I am not necessarily against a new VNIC type but we need to clarify the impact on neutron. If the new VNIC type is a variant of existing VNIC types (like normal, direct and so on) we may need to explore another option.

In addition, regarding the naming, we don't use a project name like "cybord" in our API definition. The word used should be a name which represents a correspoding functionality rather than who implements the functionality.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Akihiro:

That's my concern here. In this case [1], the Cyborg device is a smart NIC but in a future, this device could be anything else. That means the VNIC type won't define the device.

That's why I would like to talk about this topic in the Neutron meeting with Xinran.

Regards.

[1]https://specs.openstack.org/openstack/nova-specs/specs/wallaby/approved/support-sriov-smartnic.html#neutron

[Yongli He (yongli-he)]

Add @Sean Mooney @alex @Balazs Gibizer into this thread.

[Balazs Gibizer (balazs-gibizer)]

I'd like to see Sean's opinion. In my eyes what nova needs is a way to detect if a neutron port is backed by cyborg as then nova needs to talk to cyborg to get the device prepared on the cyborg side too during boot.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi Balazs:

We (almost) have this in Neutron: https://review.opendev.org/c/openstack/neutron/+/767922. When a port has a "device-profile" in the definition, this port is requesting this specific device profile from Cyborg.

As commented, a new VNIC type could not be needed.

Regards.

[Balazs Gibizer (balazs-gibizer)]

Hi Rodolfo,

So then all the combinations of existing vnic_type + cyborg device profile is allowed? E.g. vnic_type=normal + device profile or vnic_type=macvtap + device profile, etc

As far as I understand cyborg returns a PCI device address for these ports to nova and nova will add the dev to the domain as pci hostdev (Sean correct me if I'm wrong). So at least the vnic_type=normal does not make too much sense with device profile.

Cheers,
gibi

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi Balazs:

That will depend on the backend, port type, etc. But yes, I don't think we should support all combinations. Actually, as commented in the bug description, this is a kind of "direct" port. This is what I tried to explain in c#1 and c#3. We already have a "smartnic" VNIC type (for OVS using VF representors) and "direct".

In this case, "direct" + "device_profile" could be enough. And as you said, this is going to be a PCI device, that means in Neutron terms, a VNIC type "direct" port.

So maybe we can "reuse" direct for those ports instead of adding a new VNIC type. The "device_profile" descriptor could be enough, IMO.

Regards.

[Slawek Kaplonski (slaweq)]

This rfe was discussed on last drivers meeting: http://eavesdrop.openstack.org/meetings/neutron_drivers/2021/neutron_drivers.2021-01-08-14.00.log.html#l-23 and we agreed to approve it.

There was proposal to change name of the new vnic_type to something like "accelerator-<connection mechanium>" (so it would be "accelerator-direct" and "accelerator-direct-phy") - please include that proposal in the proposed patch.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

To match the current VNIC type names:
- accelerator-direct
- accelerator-direct-physical (not just "phy")

Regards.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Patch: https://review.opendev.org/c/openstack/neutron-lib/+/768324

[Yongli He (yongli-he)]

This should add to the ML2 plugin's support list, also should in the sriov-agent support list. That is the supported matrix so far by Spec.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi:

Once the n-lib with those new VNIC types is released, I'll push a patch to add them in the SR-IOV plugin.

Regards.

[Balazs Gibizer (balazs-gibizer)]

As far as I see neutron does not have the new vnic_type declared as supported by the SRIOV mechanism driver [1] (or any other driver) yet. As Feature Freeze is in ~3 days, does it mean that the support for this is deferred from Wallaby to a later release?

[1] https://github.com/openstack/neutron/blob/37f245c406768fafd9a048fa67b5ea68e7d77990/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py#L61-L63

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

You are right, this is something I should have done before. Thanks!

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.