Hugepage and KSM not configured correctly under systemd

[David Girault]

================================================================
Impact: hugepages not available for use by kvm

Fix: systemd uses the sysvinit qemu-kvm job rather than the more uptodate upstart job. To keep those in sync, the fix switches to a single common script which is run by both the upstart and sysvinit (and therefore systemd) scripts.

Test case: ls /run/hugepages

Regression potential: since init scripts are being moved around, there is a potential for this to go wrong.
================================================================

As stated by Dadio (dominique-ramaekers) in ticket #1315402:
  On 15.04 upstart isn't standard anymore...

  Now the qemu-kvm.conf upstart script isn't run.

  Shouldn't the upstart script be converted to a regular systemd script?

  Right now, the hugepages functionality isn't working anymore on Ubuntu 15.04...

Additionnaly, I saw that systemd launch the LSB job /etc/init.d/qemu-kvm instead:

dg@pc-david:/etc/init$ journalctl | grep -i qemu
juin 02 10:49:50 pc-david systemd[1]: Starting LSB: QEMU KVM module loading script...
juin 02 10:49:50 pc-david systemd[1]: Started LSB: QEMU KVM module loading script.

So, old LSB script should be removed (or disabled) when new qemy-kvm.service will be added for systemd.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dominique Ramaekers (dominique-ramaekers)]

Also more info:
Hugepages are mounted as /dev/hugepages but for qemu to work with it, it should be mounted as /run/hugepages

[Serge Hallyn (serge-hallyn)]

This should be fixed in 1:2.3+dfsg-4ubuntu1. I did not actually add a systemd unit, but rather fixed the sysv unit. On my tests systems /run/hugepages was again mounted.

[Dominique Ramaekers (dominique-ramaekers)]

Will this fix be released soon on Vivid? I've tried the updated packages from Willy installed on a Vivid host but I got permission errors trying to use hugetables (a conflict with apparmour?), so I rolled-back my system tot the 2.2 Vivid-version of qemu...

[Serge Hallyn (serge-hallyn)]

It's on my list, but since the fix in w involved moving some files between packages, it's a 'scary" SRU which will require a bunch of testing (and time).

I'm still not sure whether it is better to take the whole proper fix from w into v, or whether to just fix up the sysvinit script in v to do what is needed.

[Dominique Ramaekers (dominique-ramaekers)]

Ok, can't help you there. Anyway, thanks in advance...

[Rajkumar (kgraj2705)]

hi all,
Is this correction already available? I'm using the below version on my ubuntu 15.04, and I'm having the same issue
QEMU emulator version 2.2.0 (Debian 1:2.2+dfsg-5expubuntu9.3), Copyright (c) 2003-2008 Fabrice Bellard

[Rajkumar (kgraj2705)]

replying to my own comment,
I could see from the top of this page that, fix is already released. Could you please provide me the qemu version?
Sorry, I'm new to this page and to an extent ubuntu.

[Serge Hallyn (serge-hallyn)]

Hi,

the fix needs to go into vivid (15.04) through the SRU process, which will take some time.

[Rajkumar (kgraj2705)]

Hi Serge,
Thanks for the info. we will wait for the fix. Thanks again.

[Dominique Ramaekers (dominique-ramaekers)]

The bug also still exist in Ubuntu Willy (15.10)...

[Dominique Ramaekers (dominique-ramaekers)]

I've investigated this issue in Willy:

Hugepages are now mounted on /dev/hugepages and not in /run/hugepages

And also libvirt wants to use /dev/hugepages and more precisely /dev/hugepages/libvirt/qemu (so without the kvm in the middle of the path)

This means, the apparmor rule for huge pages points to the wrong path. The following change should be made in the profile /etc/apparmor.d/abstractions/libvirt-qemu
remove: owner "/run/hugepages/kvm/libvirt/qemu/**" rw,
add: owner "/dev/hugepages/libvirt/qemu/**" rw,

(reload the profile with service apparmor reload)

I had to create this path with the right owner-rights like this:
mkdir -p /dev/hugepages/libvirt/qemu
chown libvirt-qemu:kvm /dev/hugepages/libvirt/qemu

Now hugepages works again...

Please pay attention to the fact that in /usr/share/qemu/init/qemu-kvm-init the mountpoint /run/hugepages is created and mounted but apparently, this isn't useful any more (to my opinion)...

[Dominique Ramaekers (dominique-ramaekers)]

Weard... On my laptop I'm also on Willy. On my laptop, I only had to correct the apparmor profile...

Could some one check if it is enough to change apparmor or if it's also necessary to create the folder /dev/hugrpages/libvirt/qemu?

[Serge Hallyn (serge-hallyn)]

Quoting Dadio (<email address hidden>):
> I've investigated this issue in Willy:
>
> Hugepages are now mounted on /dev/hugepages and not in /run/hugepages

Hi,

Note that /etc/init.d/qemu-kvm calls /usr/share/qemu/init/qemu-kvm-init
which mounts hugepages under /run/hugepages.

The /dev/hugepages mounts appears to be done by systemd. We should
probably better coordinate, but filesystems really shouldn't be mounted
under /dev and we won't do this in a way that sacrifices compatibility
with other inits.

For now, setting KVM_HUGEPAGES=1 in /etc/default/qemu-kvm should
enable /run/hugepages on boot