Order of the interfaces is not honored when using virt and SR-IOV interfaces

[Itzik Brown]

When launching an instance using the following:
# nova boot --flavor m1.small --image <image id> --nic net-id=<net id> --nic port-id=<port id> vm1

Where the first interface is a non SR-IOV port and the second one is SR-IOV port the order is not preserved ,i.e. The first interface of the instance is the SR-IOV port.

Version:
openstack-nova-compute-15.0.3-3.el7ost.noarch

[Sylvain Bauza (sylvain-bauza)]

We don't really support the NIC ordering in Nova AFAIK.
If you look at https://developer.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server we just say for the 'networks' attribute that it's possible to pass multiple UUIDs, either as ports or network IDs but we don't explicitly say that the VM NICs order will be preserved.

Closing it as invalid as I don't think it's something we supported once but feel free to reopen the bug if you know that we previously supported it and we regressed somehow.

[Sylvain Bauza (sylvain-bauza)]

My fault, one just pointed me out that we actually try to preserve NIC ordering when calling Neutron :

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L2522

Looks to me like a legit bug by the fact that when the user provides a port instead of a network ID, we don't preserve the ordering since ports are treated separatly from the above.

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L841-L848

[Matt Riedemann (mriedem)]

FWIW we do preserve the order of requested networks on the boot request:

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L350

During the actual network allocation this is where we build the list of networks, either via explicitly requested networks and/or via the port IDs requested:

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L841

You can see where we don't put the port.network_id into the list of ordered_networks:

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L578

used here:

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L684

And those net_ids are what's sorted here:

https://github.com/openstack/nova/blob/286cb904055eec3525fb61b2a0f93b14448bdd3e/nova/network/neutronv2/api.py#L350

So yes we only preserve the order of requested networks, not networks and ports, or the networks associated with the ports.

[Sylvain Bauza (sylvain-bauza)]

so, after thinking more of the problem, I just feel that the current codepath is making promises that it can't sustain. To be clear, NIC ordering on a VM can sometimes be undeterministic depending on the OS.

For that precise reason, a new feature recently merged in Newton about allowing NICs to be tagged :
https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/virt-device-role-tagging.html

That way, you can query the metadata service in order to know which NIC is corresponding to which port/network you passed.

See https://developer.openstack.org/api-ref/compute/?expanded=create-server-detail and lookup for the 'networks' parameter, you'll see mentioned a 'tag' attribute that is available since 2.32 API microversion.

Closing the bug as WontFix since the community agreed on that direction.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/472733

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/472733
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7d7cc942211f0523f970dd6c1268d1d778aed8a7
Submitter: Jenkins
Branch: master

commit 7d7cc942211f0523f970dd6c1268d1d778aed8a7
Author: Sylvain Bauza <email address hidden>
Date: Fri Jun 9 17:11:24 2017 +0200

    Amend api-ref for multiple networks request

    Since there are chances that the guest OS won't honor the order of networks
    and assign NIC#1 to the second requested network, we say to our users to
    rather use device tagging and metadata querying.
    Amending the api-ref to mention explicitly that Nova doesn't support NIC
    ordering although there are codepaths for that which try that.

    Change-Id: Id134f7745111ccd12695bb041fcce182e833126a
    Related-Bug: #1696664