VF LAG not enabled due to service startup order for sriov-netplan-shim and mlnx-switchdev-mode

[Frode Nordahl]

Note that this is a bug in the mlnx-switchdev-mode and sriov-netplan-shim pacakges.

At boot you will see:
[ 275.085444] mlx5_core 0000:81:00.0: mlx5_cmd_check:769:(pid 5815): CREATE_LAG(0x840) op_mod(0x0) failed, status bad parameter(0x3), syndrome (0x7d49cb)
[ 275.100008] mlx5_core 0000:81:00.0: mlx5_create_lag:199:(pid 5815): Failed to create LAG (-22)
[ 275.109120] mlx5_core 0000:81:00.0: mlx5_activate_lag:219:(pid 5815): Failed to activate VF LAG
               Make sure all VFs are unbound prior to VF LAG activation or deactivation

We have partial fix committed here:
https://github.com/openstack-charmers/mlnx-switchdev-mode/pull/2
https://github.com/openstack-charmers/sriov-netplan-shim/pull/2

What still needs to be done is to break out the "rebind" step into a separate sub-command and run that _AFTER_ network.target is done so that VFs are bound back to the mlx5_core driver after any bonds are created.

The fix works as-is but nova-compute will complain.

[James Page (james-page)]

Testing packages will appear here:

https://launchpad.net/~openstack-charmers/+archive/ubuntu/networking-tools-testing

with the referenced pull requests

[James Page (james-page)]

Proposed updates are testing fine in our deployment.

Side effect is that nova-compute complains about missing device entries, however they do work just fine.