OOM in guest Ubuntu with inflated balloon

[AnnaMel]

There is QEMU/KVM and a Linux OS running inside the guest.
Inside the Linux guest a balloon consumes memory in accordance with
commands performed on the host side in QEMU. Rapid increases of memory consumption
inside the guest may end up with guest OOMs
since memory locked by balloon couldn’t be returned to the guest OS/vm in time.

The problem is addressed in mainstream Linux with the following patchset:

commit 5a10b7dbf904bfe01bb9fcc6298f7df09eed77d5
Author: Raushaniya Maksudova <email address hidden>
Date: Mon Nov 10 09:36:29 2014 +1030
virtio_balloon: free some memory from balloon on OOM

    Excessive virtio_balloon inflation can cause invocation of OOM-killer,
    when Linux is under severe memory pressure. Various mechanisms are
    responsible for correct virtio_balloon memory management. Nevertheless
    it is often the case that these control tools does not have enough time
    to react on fast changing memory load. As a result OS runs out of memory
    and invokes OOM-killer. The balancing of memory by use of the virtio
    balloon should not cause the termination of processes while there are
    pages in the balloon. Now there is no way for virtio balloon driver to
    free some memory at the last moment before some process will be get
    killed by OOM-killer.

    This does not provide a security breach as balloon itself is running
    inside guest OS and is working in the cooperation with the host. Thus
    some improvements from guest side should be considered as normal.

    To solve the problem, introduce a virtio_balloon callback which is
    expected to be called from the oom notifier call chain in out_of_memory()
    function. If virtio balloon could release some memory, it will make
    the system to return and retry the allocation that forced the out of
    memory killer to run.

    Allocate virtio feature bit for this: it is not set by default,
    the the guest will not deflate virtio balloon on OOM without explicit
    permission from host.

    Signed-off-by: Raushaniya Maksudova <email address hidden>
    Signed-off-by: Denis V. Lunev <email address hidden>
    Acked-by: Michael S. Tsirkin <email address hidden>
    Signed-off-by: Rusty Russell <email address hidden>

commit 1fd9c67203af91977bf3b964ff3744cf74fc6f3f
Author: Raushaniya Maksudova <email address hidden>
Date: Mon Nov 10 09:35:29 2014 +1030

    virtio_balloon: return the amount of freed memory from leak_balloon()

    This value would be useful in the next patch to provide the amount of
    the freed memory for OOM killer.

    Signed-off-by: Raushaniya Maksudova <email address hidden>
    Signed-off-by: Denis V. Lunev <email address hidden>
    CC: Rusty Russell <email address hidden>
    CC: Michael S. Tsirkin <email address hidden>
    Signed-off-by: Rusty Russell <email address hidden>

The problem is present in Ubuntu 14.10

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1587087

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tim Gardner (timg-tpi)]

Joe - please build a test kernel from 'git://kernel.ubuntu.com/rtg/ubuntu-trusty.git lts-backport-utopic-lp1587087'

[Joseph Salisbury (jsalisbury)]

I built a linux-lts-utopic test kernel, which can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1587087/

Can you test this kernel and see if it resolves this bug?

Thanks in advance!

[AnnaMel (ann-melekhova)]

Joseph, thanks kernel from http://kernel.ubuntu.com/~jsalisbury/lp1587087/ resolved bug.

[Tim Gardner (timg-tpi)]

Joe - thanks for the test kernel.

Patches submitted: https://lists.ubuntu.com/archives/kernel-team/2016-June/077944.html

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-lts-utopic - 3.16.0-77.99~14.04.1

---------------
linux-lts-utopic (3.16.0-77.99~14.04.1) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
    - LP: #1597047

  [ Josh Boyer ]

  * SAUCE: UEFI: acpi: Ignore acpi_rsdp kernel parameter when module
    loading is restricted
    - LP: #1566221
  * SAUCE: UEFI: efi: Make EFI_SECURE_BOOT_SIG_ENFORCE depend on EFI
    - LP: #1566221
  * SAUCE: UEFI MODSIGN: Import certificates from UEFI Secure Boot
    - LP: #1571691
  * SAUCE: UEFI: efi: Disable secure boot if shim is in insecure mode
    - LP: #1571691

  [ Matthew Garrett ]

  * SAUCE: UEFI: Add secure_modules() call
    - LP: #1566221
  * SAUCE: UEFI: PCI: Lock down BAR access when module security is enabled
    - LP: #1566221
  * SAUCE: UEFI: x86: Lock down IO port access when module security is
    enabled
    - LP: #1566221
  * SAUCE: UEFI: ACPI: Limit access to custom_method
    - LP: #1566221
  * SAUCE: UEFI: asus-wmi: Restrict debugfs interface when module loading
    is restricted
    - LP: #1566221
  * SAUCE: UEFI: Restrict /dev/mem and /dev/kmem when module loading is
    restricted
    - LP: #1566221
  * SAUCE: UEFI: kexec: Disable at runtime if the kernel enforces module
    loading restrictions
    - LP: #1566221
  * SAUCE: UEFI: x86: Restrict MSR access when module loading is restricted
    - LP: #1566221
  * SAUCE: UEFI: Add option to automatically enforce module signatures when
    in Secure Boot mode
    - LP: #1566221

  [ Stefan Bader ]

  * [Config] Add pm80xx scsi driver to d-i
    - LP: #1595628

  [ Tim Gardner ]

  * [Config] CONFIG_EFI_SECURE_BOOT_SIG_ENFORCE=y
  * SAUCE: UEFI: Display MOKSBState when disabled
    - LP: #1571691
  * SAUCE: UEFI: Add secure boot and MOK SB State disabled sysctl
    - LP: #1593075

  [ Upstream Kernel Changes ]

  * HID: core: prevent out-of-bound readings
    - LP: #1579190
  * mm: migrate dirty page without clear_page_dirty_for_io etc
    - LP: #1581865
    - CVE-2016-3070
  * virtio_balloon: return the amount of freed memory from leak_balloon()
    - LP: #1587087
  * virtio_balloon: free some memory from balloon on OOM
    - LP: #1587087

 -- Kamal Mostafa <email address hidden> Tue, 28 Jun 2016 11:43:10 -0700