[Ubuntu-24.04] Hugepage memory is not getting released even after destroying the guest!

[bugproxy]

---Problem Description---

Hugepages memory is not getting released even after destroying the guest

Machine Type = P10 Denali LPAR

---uname output---
Linux ubuntu2404lp3 6.8.0-22-generic #22-Ubuntu SMP Thu Apr 4 22:47:57 UTC 2024 ppc64le ppc64le ppc64le GNU/Linux

---Steps to Reproduce---
1. Create a guest which is backed by hugepages.
2. Destroy the guest
3. execute "free -h" or "cat /proc/meminfo" to see that Hugepage memory is still getting held.

HugePages_Total: 20480
HugePages_Free: 20419
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 41943040 kB
DirectMap4k: 0 kB
DirectMap64k: 52428800 kB
DirectMap2M: 0 kB
DirectMap1G: 0 kB
root@ubuntu2404lp3:~# virsh list --all
 Id Name State
---------------------------
 - ramlp2g1 shut off
 - ramlp2g2 shut off
 - ramlp2g3 shut off
 - ramlp3g3 shut off

root@ubuntu2404lp3:~# free -h
               total used free shared buff/cache available
Mem: 48Gi 43Gi 4.6Gi 2.6Mi 277Mi 4.6Gi
Swap: 8.0Gi 243Mi 7.8Gi
root@ubuntu2404lp3:~#

====
This is an issue created by commit 1b151e2435fc ("block: Remove special-casing of compound pages") that moved the direct-io hugetlb handling from compound pages to folios.

Following commit has been proposed and merged into 6.9-rc1 which fixes this issue.
38b43539d64b2fa020b3b9a752a986769f87f7a6("block: Fix page refcounts for unaligned buffers in __bio_release_pages()")

So the same needs to be backported to the Ubuntu24.04 kernel as well.

[Frank Heimes (fheimes)]

I just had a look at noble's master-next tree:
$ git remote get-url origin
git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble
$ git branch
* master-next

and found that commit 19c1ceeca8ed is already applied:
$ git log --oneline | grep -n "block: Fix page refcounts for unaligned buffers in __bio_release_pages"
300:19c1ceeca8ed block: Fix page refcounts for unaligned buffers in __bio_release_pages()

but not yet tagged with (incl. in) any version:
$ git tag --contains 19c1ceeca8ed
$

This is probably because it just came in via upstream stable updates:
$ git show 19c1ceeca8ed | sed '/^diff --git/q'
commit 19c1ceeca8ed31d207859e659482ee83ea1959f5
Author: Tony Battersby <email address hidden>
Date: Thu Feb 29 13:08:09 2024 -0500

    block: Fix page refcounts for unaligned buffers in __bio_release_pages()

    BugLink: https://bugs.launchpad.net/bugs/2060531

    [ Upstream commit 38b43539d64b2fa020b3b9a752a986769f87f7a6 ]

    Fix an incorrect number of pages being released for buffers that do not
    start at the beginning of a page.

    Fixes: 1b151e2435fc ("block: Remove special-casing of compound pages")
    Cc: <email address hidden>
    Signed-off-by: Tony Battersby <email address hidden>
    Tested-by: Greg Edwards <email address hidden>
    Link: https://<email address hidden>
    Signed-off-by: Jens Axboe <email address hidden>
    Signed-off-by: Sasha Levin <email address hidden>
    Signed-off-by: Paolo Pisati <email address hidden>

diff --git a/block/bio.c b/block/bio.c

And since the lastest tagged Ubuntu kernel version in master-next is: Ubuntu-6.8.0-31.31
$ git log --oneline | grep -n $(git describe --tags --abbrev=0)
1170:7fdb45c9bbbc (tag: Ubuntu-6.8.0-31.31) UBUNTU: Ubuntu-6.8.0-31.31

commit 19c1ceeca8ed will be automatically part of the next/upcoming Ubuntu kernel (> Ubuntu-6.8.0-31.31).

With that I'm updating the status of this bug to Fix Committed.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2024-05-06 06:50 EDT-------
I have verified this bug on the latest available Ubuntu 24.04 host. The issue has been fixed and is not getting reproduced anymore!

kernel: 6.8.0-31-generic #31-Ubuntu

Before guest creation:

HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

After guest creation:

HugePages_Total: 20480
HugePages_Free: 15360
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

After destroying the Before guest:

HugePages_Total: 20480
HugePages_Free: 20480
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the linux/6.8.0-32.32 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux' to 'verification-done-noble-linux'. If the problem still exists, change the tag 'verification-needed-noble-linux' to 'verification-failed-noble-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2024-05-06 14:39 EDT-------
Frank has confirmed in his previous comment that the fix will be part kernel 6.8.0-31.31.

I also verified the bug on the same level of kernel and updated my comments above that the fix is working as expected.

Since it is a generic kernel, I believe that the Fix is already a part of the source code. If not, please get this included in the upcoming release.

*** I have verified the Fix and is working as expected ***

[Frank Heimes (fheimes)]

Many thanks for your confirmation Kowshik Jois.
With that I'm updating the tags accordingly (and set them to verification done).