libvirt raise Could not access KVM kernel module error

[Jeffrey Zhang]

When deploying on arm64, permission denied reported:
Terminating instance: libvirtError: internal error: process exited while connecting to monitor: Could not access KVM kernel module: Permission denied

Also hitting this on tripleo x86_64 deployments since https://review.openstack.org/492780 merged

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501909
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=b0e26030583f881934a42e6c5f121736f9f5341e
Submitter: Jenkins
Branch: stable/pike

commit b0e26030583f881934a42e6c5f121736f9f5341e
Author: Jeffrey Zhang <email address hidden>
Date: Wed Sep 6 17:59:39 2017 +0800

    Revert "nova-libvirt: fix kvm permission issue"

    This reverts commit 43650d5837372449294a7a22e9c9167a92744e96.

    This fix do not work with CentOS x86_64. Since I haven't arm64 env, and
    we have fix this soon before pike is release. Just revert this and fix
    in the future.

    Change-Id: Id9eb531de7d05051f38e3ed13b64ae7abf552767
    Partial-Bug: #1715356
    (cherry picked from commit e9929dae60033fda9426671628b8c874dde97480)

[Oliver Walsh (owalsh)]

This needs to be mode 666 - there is a udev rule that does this:

[heat-admin@overcloud-novacomputebm-0 ~]$ rpm -qf /lib/udev/rules.d/80-kvm.rules
qemu-kvm-ev-2.6.0-28.el7.10.1.x86_64
[heat-admin@overcloud-novacomputebm-0 ~]$ cat /lib/udev/rules.d/80-kvm.rules
KERNEL=="kvm", GROUP="kvm", MODE="0666"

[Oliver Walsh (owalsh)]

For reference this is the correct behaviour on Fedora/CentOS/RHEL: https://bugzilla.redhat.com/show_bug.cgi?id=497341

Also it's being moved to systemd from the qemu rpm, so I think we still need to mirror the behaviour in kolla - https://bugzilla.redhat.com/show_bug.cgi?id=1431876, which also mentions that Debian sets it to 0660 + group kvm.

[Steven Dake (sdake)]

Disagree on 666. Just because udev does it, doesn't mean it is correct. E.G. dd if=/dev/zero of=/dev/kvm will blow up the system. kvm should only be accessible by the qemu group. It is possible to properly set permissions for qemu:

e.g.:
https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/templates/qemu.conf.j2#L10-L11

[Oliver Walsh (owalsh)]

Clearly there isn't consensus on what it correct. The Fedora Virtualization Maintainers decided 0666, the Debian maintainers decided 0660. I think the respective mailing lists and/or bug trackers is a more appropriate place to discuss it.

I don't agree with kolla being opinionated on matters such as this. If it diverges from the expected behaviour it introduces subtle and confusing bugs, such as this.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/501653
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=f1b98a4925826333cf22dafe15ef755066c51d48
Submitter: Zuul
Branch: master

commit f1b98a4925826333cf22dafe15ef755066c51d48
Author: Kevin Zhao <email address hidden>
Date: Thu Sep 7 17:28:29 2017 +0800

    nova-libvirt: fix kvm permission issue

    When deploy in AArch64, will meet kvm permission issue.
    Fix it with Arch specified.

    Closes-bug: #1715356

    Change-Id: I09dc27693a83dc77704773a25547725de480dbe8
    Signed-off-by: Kevin Zhao <email address hidden>

[xinliang (xin3liang)]

The root cause for this issue is that qemu not run with nova user on debian.
If we set qemu running with nova user with this patch: https://review.openstack.org/#/c/525891/
Then we need to revert commit f1b98a4925826333cf22dafe15ef755066c51d48.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/525900

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 6.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525900
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=14cda91c4ef7020ac43f430869798387de15dae4
Submitter: Zuul
Branch: master

commit 14cda91c4ef7020ac43f430869798387de15dae4
Author: Xinliang Liu <email address hidden>
Date: Wed Dec 6 14:23:58 2017 +0800

    Revert "nova-libvirt: fix kvm permission issue"

    This is not needed. If we make sure qemu use nova user.
    Because nova user in group qemu.

    See comment #7 of bug #1715356:
    The root cause for this issue is that qemu not run with nova user on
    debian.
    If we set qemu running with nova user with this patch:
    https://review.openstack.org/#/c/525891/
    Then we need to revert commit f1b98a4925826333cf22dafe15ef755066c51d48.

    This reverts commit f1b98a4925826333cf22dafe15ef755066c51d48.
    Closes-bug: #1715356
    Depends-on: I36720af0c7d3dd7c69d2404843f54c0991aea1bb

    Change-Id: I62fbcf9e4ee5c3170c96576698f4ae8b66db1b74

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 6.0.0.0b3 development milestone.