Hide "ballooned" memory from /proc in guest Ubuntu

[AnnaMel]

There is QEMU/KVM and a Linux guest running inside the guest. The amount
of memory available for guest could be adjusted by balloon for better
host scalability. The problem that this change is visible for end-user
actually using the guest. This could (potentially) result in lawsuite
from the end-user to hosting provides.

The problem is addressed in mainstream Linux with the following patch set:

commit 997e120843e82609c8d99a9d5714e6cf91e14cbe
Author: Denis V. Lunev <email address hidden>
Date: Thu Aug 20 00:49:49 2015 +0300
virtio_balloon: do not change memory amount visible via /proc/meminfo

    Balloon device is frequently used as a mean of cooperative memory control
    in between guest and host to manage memory overcommitment. This is the
    typical case for any hosting workload when KVM guest is provided for
    end-user.

    Though there is a problem in this setup. The end-user and hosting provider
    have signed SLA agreement in which some amount of memory is guaranted for
    the guest. The good thing is that this memory will be given to the guest
    when the guest will really need it (f.e. with OOM in guest and with
    VIRTIO_BALLOON_F_DEFLATE_ON_OOM configuration flag set). The bad thing
    is that end-user does not know this.

    Balloon by default reduce the amount of memory exposed to the end-user
    each time when the page is stolen from guest or returned back by using
    adjust_managed_page_count and thus /proc/meminfo shows reduced amount
    of memory.

    Fortunately the solution is simple, we should just avoid to call
    adjust_managed_page_count with VIRTIO_BALLOON_F_DEFLATE_ON_OOM set.

    Signed-off-by: Denis V. Lunev <email address hidden>
    CC: Michael S. Tsirkin <email address hidden>
    Signed-off-by: Michael S. Tsirkin <email address hidden>

commit b4d34037329f46ed818d3b0a6e1e23b9c8721f79
Author: Denis V. Lunev <email address hidden>
Date: Thu Aug 20 00:49:48 2015 +0300
virtio_ballon: change stub of release_pages_by_pfn

    and rename it to release_pages_balloon. The function originally takes
    arrays of pfns and now it takes pointer to struct virtio_ballon.
    This change is necessary to conditionally call adjust_managed_page_count
    in the next patch.

    Signed-off-by: Denis V. Lunev <email address hidden>
    CC: Michael S. Tsirkin <email address hidden>
    Signed-off-by: Michael S. Tsirkin <email address hidden>

The issue affects ubuntu_server_14.0, ubuntu_server_15.04

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1587089

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tim Gardner (timg-tpi)]

Joe - please build a test kernel from 'git://kernel.ubuntu.com/rtg/ubuntu-wily.git lp1587089'

[Joseph Salisbury (jsalisbury)]

I built a wily test kernel, which can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1587089

Can you test this kernel and see if it resolves this bug?

Thanks in advance!

[AnnaMel (ann-melekhova)]

Joseph, thanks!
Kernel from http://kernel.ubuntu.com/~jsalisbury/lp1587089 resolved bug for Wily, but this bug also affects Trusty, could you please port patches to it too?

[Tim Gardner (timg-tpi)]

Joe, please build test kernels from these 3 branches:

git://kernel.ubuntu.com/rtg/ubuntu-trusty.git lp1587089
git://kernel.ubuntu.com/rtg/ubuntu-trusty.git lts-backport-utopic-lp1587089
git://kernel.ubuntu.com/rtg/ubuntu-vivid.git lp1587089

[Joseph Salisbury (jsalisbury)]

A vivid test kernel is available. It can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1587091/vivid/

The trusty and lts-u test kernels failed. It appears some prereq commits are required. Those test kernels will be available shortly.

[Joseph Salisbury (jsalisbury)]

A trusty test kernel is also now available for testing. It can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1587089/trusty/

[AnnaMel (ann-melekhova)]

Joseph, thanks!
I've tested http://kernel.ubuntu.com/~jsalisbury/lp1587089/trusty/, it resolved bug for Trusty.
Unfortunately I don't know how to test Vivid, since there are no installation images of it.

[Tim Gardner (timg-tpi)]

https://lists.ubuntu.com/archives/kernel-team/2016-June/078529.html

[Kamal Mostafa (kamalmostafa)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[AnnaMel (ann-melekhova)]

It looks like kernel from trusty-proposed doesn't solve neither this problem nor https://bugs.launchpad.net/ubuntu/+source/linux-lts-utopic/+bug/1587087 (if set ~100 MB, kernel doesn't panic, but system "freezes"). What kind of information can help you? Is it right kernel?

user@ubuntu:~$ uname -a
Linux ubuntu 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

user@ubuntu:~$ free -h
             total used free shared buffers cached
Mem: 993M 119M 873M 412K 24M 46M
-/+ buffers/cache: 49M 943M
Swap: 1.0G 0B 1.0G
user@ubuntu:~$ ssh root@host-server virsh setmem ubuntu-14.04.4 --size 800M

user@ubuntu:~$ free -h
             total used free shared buffers cached
Mem: 769M 126M 643M 412K 24M 52M
-/+ buffers/cache: 49M 719M
Swap: 1.0G 0B 1.0G

[Tim Gardner (timg-tpi)]

AnnaMel - the kernel you are testing in #11 doesn't look like the right one. Please make sure you have installed and booted the Trusty kernel from proposed:

https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/10184623/+files/linux-image-3.13.0-92-generic_3.13.0-92.139_amd64.deb
https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/10184623/+files/linux-image-extra-3.13.0-92-generic_3.13.0-92.139_amd64.deb

According to your comment in #8 this kernel should solve your problem.

[AnnaMel (ann-melekhova)]

Tim, I've checked with kernel 3.13.0-92.139 from "proposed" and it solved problem.
Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.13.0-92.139

---------------
linux (3.13.0-92.139) trusty; urgency=low

  [ Kamal Mostafa ]

  * Release Tracking Bug
    - LP: #1597060

  [ Josh Boyer ]

  * SAUCE: UEFI: acpi: Ignore acpi_rsdp kernel parameter when module
    loading is restricted
    - LP: #1566221
  * SAUCE: UEFI: efi: Make EFI_SECURE_BOOT_SIG_ENFORCE depend on EFI
    - LP: #1566221
  * SAUCE: UEFI MODSIGN: Import certificates from UEFI Secure Boot
    - LP: #1566221, #1571691
  * SAUCE: UEFI: efi: Disable secure boot if shim is in insecure mode
    - LP: #1566221, #1571691

  [ Matthew Garrett ]

  * SAUCE: UEFI: Add secure_modules() call
    - LP: #1566221
  * SAUCE: UEFI: PCI: Lock down BAR access when module security is enabled
    - LP: #1566221
  * SAUCE: UEFI: x86: Lock down IO port access when module security is
    enabled
    - LP: #1566221
  * SAUCE: UEFI: ACPI: Limit access to custom_method
    - LP: #1566221
  * SAUCE: UEFI: asus-wmi: Restrict debugfs interface when module loading
    is restricted
    - LP: #1566221
  * SAUCE: UEFI: Restrict /dev/mem and /dev/kmem when module loading is
    restricted
    - LP: #1566221
  * SAUCE: UEFI: kexec: Disable at runtime if the kernel enforces module
    loading restrictions
    - LP: #1566221
  * SAUCE: UEFI: x86: Restrict MSR access when module loading is restricted
    - LP: #1566221
  * SAUCE: UEFI: Add option to automatically enforce module signatures when
    in Secure Boot mode
    - LP: #1566221

  [ Stefan Bader ]

  * [Config] Add pm80xx scsi driver to d-i
    - LP: #1595628

  [ Tim Gardner ]

  * [Config] CONFIG_EFI_SECURE_BOOT_SIG_ENFORCE=y
  * SAUCE: UEFI: Display MOKSBState when disabled
    - LP: #1566221, #1571691
  * SAUCE: UEFI: Add secure boot and MOK SB State disabled sysctl
    - LP: #1593075
  * SAUCE: UEFI: Set EFI_SECURE_BOOT bit in x86_efi_facility
    - LP: #1593075
  * [Config] CONFIG_EFI=n for arm64
    - LP: #1566221

  [ Upstream Kernel Changes ]

  * powerpc/tm: Abort syscalls in active transactions
    - LP: #1572624
  * HID: core: prevent out-of-bound readings
    - LP: #1579190
  * efi: Add separate 32-bit/64-bit definitions
    - LP: #1566221
  * x86/efi: Build our own EFI services pointer table
    - LP: #1566221
  * mm: migrate dirty page without clear_page_dirty_for_io etc
    - LP: #1581865
    - CVE-2016-3070
  * oom_kill: change oom_kill.c to use for_each_thread()
    - LP: #1592429
  * oom_kill: has_intersects_mems_allowed() needs rcu_read_lock()
    - LP: #1592429
  * oom_kill: add rcu_read_lock() into find_lock_task_mm()
    - LP: #1592429
  * virtio_balloon: return the amount of freed memory from leak_balloon()
    - LP: #1587089
  * virtio_balloon: free some memory from balloon on OOM
    - LP: #1587089
  * virtio_ballon: change stub of release_pages_by_pfn
    - LP: #1587089
  * virtio_balloon: do not change memory amount visible via /proc/meminfo
    - LP: #1587089

 -- Kamal Mostafa <email address hidden> Tue, 28 Jun 2016 12:40:49 -0700