Permissions of /dev/kvm are altered by Ubuntu 16.04 aarch64 udev rules and nova_libvirt/nova_compute can't access the kvm module

We are using kolla 5.0.2 (with custom build images ranging from several month to a few days ago based on Ubuntu 16.04) deployed on Ubuntu 16.04 hosts. We run a mixed cluster of x86 and aarch64 hosts and we noticed that some time in the last months the behaviour between the two architectures changed.

On aarch64, the starting of libvirtd --listen in the nova_libvirt container triggers the kernel udev rules:

cat /lib/udev/rules.d/60-qemu-system-common.rules
KERNEL=="kvm", GROUP="kvm", MODE="0660"

As a result, /dev/kvm is then owned by the kvm group on the host (GID 126) and not by the qemu group of kolla (GID 42427)

Somehow, the udev rule is not trigger on x86_64

We found two work arounds so far:

Create a custom udev rule:

cat /etc/udev/rules.d/60-qemu-system-common.rules
KERNEL=="kvm", GROUP="42427", MODE="0660"
cat /lib/udev/rules.d/60-qemu-system-common.rules
KERNEL=="kvm", GROUP="kvm", MODE="0660"

and apply it:
udevadm control --reload-rules

Alternatively, add a custom group to the nova_libvirt and nova_compute container in /etc/group:
kvm-hypervisor:x:126:nova

Both solutions are not brilliant...