Deploying Microk8s charm on LXD KVMs fails

[Giuseppe Petralia]

This is happening in juju 3.1.2 deploying KVMs on top of an LXD cluster.

# Steps to reproduce:

1. Add machine

juju add-machine --constraints="virt-type=virtual-machine mem=4G cores=4 root-disk=40G root-disk-source=remote" --base ubuntu@22.04

2. When machine is ready deploy Microk8s on it with:

juju add-unit microk8s --to <id-of-the-machine>

# Expected result:

Unit starts installing on the machine

# Actual result:

Machine goes to error state with message:

19 error 172.16.0.16 juju-657122-19 ubuntu@22.04 cannot upgrade machine's lxd profile: update failed: Key "linux.kernel_modules" cannot be updated when VM is running

# Workaround:

Stop the VM and then start with:

lxc stop --force juju-657122-19
lxc start juju-657122-19

and machine will no longer be in error state and once the KVM is running again unit will start installing on it.

[Giuseppe Petralia (peppepetra)]

The problem is juju blindly applying lxd-profile.yaml [0] shipped by the charm even if this does not make sense for a KVM (even if on top of LXD)

The solution can be either:

a) ignore lxd-profile.yaml if machine constraints have `virt-type=virtual-machine`

b) use a separate lxd profile like lxd-vm-profile.yaml to be used for LXDs with constraint `virt-type=virtual-machine`

A way to reproduce without juju is

```
lxc launch ubuntu:jammy v1 --vm
lxc profile set default linux.kernel_modules=foo
Error: The following instances failed to update (profile change still saved):
 - Project: default, Instance: v1: Key "linux.kernel_modules" cannot be updated when VM is running
echo $?
1
```

Links:
0. https://github.com/canonical/charm-microk8s/blob/master/lxd-profile.yaml

[Giuseppe Petralia (peppepetra)]

Filed a bug with lxd as well https://github.com/lxc/lxd/issues/11888

[Heather Lanigan (hmlanigan)]

The juju charm lxd profile work was implemented well before juju supported vm's via lxd, thus does not distinguish. Looks this needs to be investigated.

There is no expectation in the juju lxd profile implementation that a container may require a reboot. In fact it's not something that we'd want to do. Charms with lxd profiles can be deployed to a new or existing lxd container at any time with no guarantee that it has no other units running on it.

[Trent Lloyd (lathiat)]

I'm hitting this too when deploying openstack.

Looks like LXD has committed a fix but it's not yet in 5.15 (latest/stable).