allow juju to create SRIOV instances on openstack

[Junien F]

Hi,

If you want to use an SRIOV instance on Openstack with Juju currently, you have to create the ports and instance manually, and then add it to a model via the "manual" provider ("juju add-machine ssh:${IP}").

It would be nice if Juju was able to launch such instances on its own.
The steps are documented at https://docs.openstack.org/neutron/ussuri/admin/config-sriov.html#launching-instances-with-sr-iov-ports

Thanks !

[James Troup (elmo)]

BootStack has customers using Juju on top of their OpenStack who have also run into this limitation, FWIW

[John A Meinel (jameinel)]

So that link describes a bunch of steps about creating a distinct network for sriov and other configuration. Is that part of what you're doing, or is it just the `openstack create port --vnic-type 'direct'` that you would want to automate?
There also seems to be direct-physical as well as direct.

How would this be modeled on the Juju side? How would we know when to use sr-iov vs normal? Is it a different space, such that all things in that space use sr-iov devices?
Is it a model global that we would want to do it on all instances in the model?
Is it a 'machine constraint' so that a given application makes requests to openstack with the additional fields?
We have had syntax for custom storage requests (provisioned IOPS, etc), we haven't done the equivalent for networking.

I think the actual implementation of creating the port and having that work is relatively straightforward, but figuring out a nice way to model it so that you can express what you want in a good syntax is the tricky bit.

[Junien F (axino)]

I can only speak for my own (limited) usage, but :

- I think it's fine to automate just the port creation. The network creation is a one-time thing (again, for my use case)

- On the Juju side, I think it makes sense to make it a different space

- I'd make it a "machine constraint" I think ? I can definitely see cases where it's not required on all machines of a model. But again, CMR exists so models can be split etc...

[James Troup (elmo)]

[Junien responded to most of your comments; I skipped

John A Meinel <email address hidden> writes:

> How would we know when to use sr-iov vs normal?

SR-IOV imposes additional burdens on the guest (for example, the VM
needs to be able to drive the NIC itself), so using SR-IOV for a VM
would have to be opt-in.

> Is it a different space, such that all things in that space use
> sr-iov devices?

This wouldn't work for us as we have customers who mix and match
SR-IOV and virtio (normal) VMs in the same space/subnet.

> Is it a model global that we would want to do it on all instances in
> the model?

I don't see this is a use case in the first instance.

> Is it a 'machine constraint' so that a given application makes
> requests to openstack with the additional fields?

It could be yes.

--
James

[Joseph Phillips (manadart)]

Since we added multi-NIC support to O7k, whenever we detect space constraints the instance ports are created manually and then attached, so we already have an explicit port creation logical path here.

This might be easy to do via machine constraints.

[John A Meinel (jameinel)]

So the issue that I see with this discussion is that "for some guests" in
this space, they need to use SR-IOV, but not for all. I guess the other
question is whether "all units of this application" must use SR-IOV?
That does fit a constraints model. But does that also apply for generic
substrates (vs just on Openstack).
And given multiple network interfaces, how would you define that interface
X needs SR-IOV but not Y.
(possibly it is a list of spaces that should use SR-IOV for this
application)

On Tue, Jul 20, 2021 at 3:55 AM Joseph Phillips <email address hidden>
wrote:

> Since we added multi-NIC support to O7k, whenever we detect space
> constraints the instance ports are created manually and then attached,
> so we already have an explicit port creation logical path here.
>
> This might be easy to do via machine constraints.
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1933255
>
> Title:
> allow juju to create SRIOV instances on openstack
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1933255/+subscriptions
>
>

[Junien F (axino)]

I'd expect all units of an application to require SRIOV yes.

[Haw Loeung (hloeung)]

Yeah, I think this should be applied to all units of an application. If we wanted non-SR-IOV units, we could deploy those as separate applications. This also prevents any accidental non-SR-IOVs being provisioned.

[Nikolay Vinogradov (nikolay.vinogradov)]

Once this is fixed, it would be significantly simpler to deploy Kubernetes on top of OpenStack for the use cases that need network acceleration (such as telco deployments). Currently there is no way to deploy the VM on top of OpenStack with SR-IOV NIC with Juju, neither it can be done post-deployment by adding SR-IOV port to the running or stopped instance (e.g. if we wanted to add it post-deployment to a Juju-provisioned instance), so we have to use manual machines to overcome this.

[Canonical Juju QA Bot (juju-qa-bot)]

This Medium-priority bug has not been updated in 60 days, so we're marking it Low importance. If you believe this is incorrect, please update the importance.

[Joseph Phillips (manadart)]

This is being mooted for our next cycle (24.04).