Realtime feature mlockall: Cannot allocate memory
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Cloud Archive |
Fix Released
|
Medium
|
Unassigned | ||
Mitaka |
Fix Released
|
Medium
|
Jorge Niedbalski | ||
Ocata |
Fix Released
|
Medium
|
Jorge Niedbalski | ||
libvirt (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Xenial |
Fix Released
|
Medium
|
Jorge Niedbalski | ||
Zesty |
Fix Released
|
Medium
|
Jorge Niedbalski | ||
Artful |
Fix Released
|
Medium
|
Unassigned |
Bug Description
[Impact]
* Guest definitions that uses locked memory + hugepages fail to spawn
* Backport upstream fix to solve that issue
[Environment]
root@buneary:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial
root@buneary:~# uname -r
4.10.0-29-generic
Reproducible also with the 4.4 kernel.
[Detailed Description]
When a guest memory backing stanza is defined using the <locked/> stanza + hugepages,
as follows:
<memoryBacking>
<hugepages>
<page size='1' unit='GiB' nodeset='0'/>
<page size='1' unit='GiB' nodeset='1'/>
</hugepages>
<nosharedpa
<locked/>
</memoryBacking>
(Full guest definition: http://
The guest fails to start due to the following error:
2017-08-02 20:25:03.714+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.12 (Christian Ehrhardt <email address hidden> Wed, 19 Jul 2017 08:28:14 +0200), qemu version: 2.5.0 (Debian 1:2.5+dfsg-
LC_ALL=C PATH=/usr/
Domain id=14 is tainted: host-cpu
char device redirected to /dev/pts/1 (label charserial0)
mlockall: Cannot allocate memory
2017-08-
2017-08-02 20:25:37.811+0000: shutting down
This seems to be due to the setrlimit for RLIMIT_MEMLOCK is too low for mlockall
to work given the large amount of memory.
There is a libvirt upstream patch that enforces the existence of the
hard_limit stanza when using with <locked/> in the memory backing settings.
https:/
Memory locking can only work properly if the memory locking limit
for the QEMU process has been raised appropriately: the default one
is extremely low, so there's no way the guest will fit in there.
The commit https:/
[Test Case]
* Define a guest that uses the following stanzas (See for a full guest reference: http://
<memory unit='GiB'
<currentMemory unit='GiB'
<memoryBacking>
<hugepages>
<page size='1' unit='GiB' nodeset='0'/>
<page size='1' unit='GiB' nodeset='1'/>
</hugepages>
<nosharedpa
<locked/>
</memoryBacking>
* virsh define guest.xml
* virsh start guest.xml
* Without the fix, the following error will be raised and the guest
will not start.
root@buneary:
error: Failed to start domain reproducer2
error: internal error: process exited while connecting to monitor: mlockall: Cannot allocate memory
2017-08-
* With the fix, the error shouldn't be displayed and the guest started
[Suggested Fix]
* https:/
[Regression Potential]
* There is one (theoretical) thing to think of - the change increases
the lock resource limits for the spawned qemu. Maybe that could be
used as an attack. Fortunately the defnition does have neither
locked nor hugepages by default so opt-in, and while the guest can
do all kind of things when exploited it can only hardly change it's
(virtual) physical memory to increase what the host might be locking.
Yet worth to mention IMHO
* The general regression is rather low as I said "locked" is not
default and the code path is only affecting domains configured that
way. That makes the change rather safe for the overall user - and the
one using locked likely need it.
tags: | added: canonical-bootstack |
Changed in libvirt (Ubuntu Artful): | |
importance: | Undecided → Low |
Changed in libvirt (Ubuntu Artful): | |
status: | Confirmed → In Progress |
importance: | Low → High |
importance: | High → Medium |
assignee: | nobody → Jorge Niedbalski (niedbalski) |
description: | updated |
Changed in libvirt (Ubuntu Artful): | |
assignee: | Jorge Niedbalski (niedbalski) → nobody |
Changed in libvirt (Ubuntu Artful): | |
status: | In Progress → Confirmed |
Changed in libvirt (Ubuntu Xenial): | |
status: | New → In Progress |
Changed in libvirt (Ubuntu Zesty): | |
status: | New → In Progress |
Changed in libvirt (Ubuntu Xenial): | |
importance: | Undecided → High |
importance: | High → Medium |
Changed in libvirt (Ubuntu Zesty): | |
importance: | Undecided → Medium |
Changed in libvirt (Ubuntu Xenial): | |
assignee: | nobody → Jorge Niedbalski (niedbalski) |
Changed in libvirt (Ubuntu Zesty): | |
assignee: | nobody → Jorge Niedbalski (niedbalski) |
description: | updated |
Changed in cloud-archive: | |
status: | New → Fix Committed |
importance: | Undecided → Medium |
tags: |
added: verification-done verification-done-zesty removed: verification-needed verification-needed-zesty |
Changed in cloud-archive: | |
status: | Fix Committed → Fix Released |
tags: |
added: sts-sru-done removed: sts-sru-needed |
Status changed to 'Confirmed' because the bug affects multiple users.