"numastat" doesn't display correct information for the guest.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Fix Released
|
High
|
Canonical Server | ||
numactl (Ubuntu) |
Fix Released
|
High
|
Ubuntu on IBM Power Systems Bug Triage | ||
Bionic |
Fix Released
|
High
|
Unassigned | ||
Cosmic |
Fix Released
|
High
|
Unassigned |
Bug Description
[Impact]
* Some less common numa hierarchies break numactl
* backport upstream change to fix that
- https:/
[Test Case]
* You have to create an "affected" Numa topology
E.g. on a full mem stacked P9 create a guest like:
<vcpu placement='static' current=
<cputune>
<vcpupin vcpu='0' cpuset='64'/>
<vcpupin vcpu='1' cpuset='65'/>
<vcpupin vcpu='2' cpuset='66'/>
<vcpupin vcpu='3' cpuset='67'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='8'/>
</numatune>
* Then after starting that numactl will fail to show it's data, for
example:
# numastat -c qemu-system-ppc64
Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc)
Node 0 Node 8 ...
------ ------ ...
Huge 0 0 ...
Heap 0 0 ...
Stack 14 0 ...
Private 936 0 ...
------- ------ ------ ...
Total 936 0 ...
(more details on the Host used below)
[Regression Potential]
* The change is rather small just skipping nodes not matching the
expected numbers (important to not crash). If anything then I can
think of a regression where it would skip a numa node that it
would need to access. But I have seen none such case in my tests, so
I hope it will be fine.
[Other Info]
* n/a
---
== Comment: #0 - SANTWANA SAMANTRAY <email address hidden> - 2019-02-20 23:48:43 ==
---Problem Description---
numastat doesn't display correct information for kvm guests.
The guest is configured with vcpupin from the node8:
Snippet of the guest XML :
<vcpu placement='static' current=
<cputune>
<vcpupin vcpu='0' cpuset='64'/>
<vcpupin vcpu='1' cpuset='65'/>
<vcpupin vcpu='2' cpuset='66'/>
<vcpupin vcpu='3' cpuset='67'/>
<emulatorpin cpuset='64-67'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='8'/>
</numatune>
== Host Details ==
# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 4
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 6
Model: 2.2 (pvr 004e 1202)
Model name: POWER9, altivec supported
CPU max MHz: 3800.0000
CPU min MHz: 2300.0000
L1d cache: 32K
L1i cache: 32K
L2 cache: 512K
L3 cache: 10240K
NUMA node0 CPU(s): 0-63
NUMA node8 CPU(s): 64-127
NUMA node252 CPU(s):
NUMA node253 CPU(s):
NUMA node254 CPU(s):
NUMA node255 CPU(s):
After starting the guest, the numastat doesn't display the guest process to be running in node8.
# numastat -c qemu-system-ppc64
Per-node process memory usage (in MBs) for PID 5738 (qemu-system-ppc)
Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total
------ ------ -------- -------- -------- -------- -----
Huge 0 0 0 0 0 0 0
Heap 0 0 0 0 0 0 14
Stack 14 0 0 0 0 0 0
Private 936 0 0 0 0 0 3064
------- ------ ------ -------- -------- -------- -------- -----
Total 936 0 0 0 0 0 3079
# service numad status
* numad.service - numad - The NUMA daemon that manages application locality.
Loaded: loaded (/lib/systemd/
Active: active (running) since Thu 2019-02-21 00:43:54 EST; 20s ago
Docs: man:numad
Process: 8091 ExecStart=
Main PID: 8092 (numad)
Tasks: 2 (limit: 19660)
CGroup: /system.
`-8092 /usr/bin/numad -i 15
Feb 21 00:43:54 ltcgen3 systemd[1]: Starting numad - The NUMA daemon that manages application locality....
Feb 21 00:43:54 ltcgen3 systemd[1]: Started numad - The NUMA daemon that manages application locality..
---uname output---
Linux ltcgen3 4.15.0-1016-ibm-gt #18-Ubuntu SMP Thu Feb 7 16:58:31 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux
Machine Type = Witherspoon
== Versions Installed ==
qemu 1:2.11+
qemu-kvm 1:2.11+
qemu-system-ppc 1:2.11+
libvirt0:ppc64el 4.0.0-1ubuntu8.6
libnuma-dev:ppc64el 2.0.11-2.1
libnuma1:ppc64el 2.0.11-2.1
numactl 2.0.11-2.1
numad 0.5+20150602-5
---Debugger---
A debugger is not configured
---Steps to Reproduce---
1. Configure the guest with vcpu pinning and memory binding to one of the numa node.
<cputune>
<vcpupin vcpu='0' cpuset='64'/>
<vcpupin vcpu='1' cpuset='65'/>
<vcpupin vcpu='2' cpuset='66'/>
<vcpupin vcpu='3' cpuset='67'/>
<emulatorpin cpuset='64-67'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='8'/>
</numatune>
2. Start the guest.
3. Check the "numastat" for the guest details.
4. The output doesn't display any data for desired node.
Contact Information = Santwana <email address hidden>
== Comment: #1 - SANTWANA SAMANTRAY <email address hidden> - 2019-02-20 23:49:58 ==
== Comment: #4 - SEETEENA THOUFEEK <email address hidden> - 2019-02-22 00:40:37 ==
I am not able to replicate in upstream code
We need to cherry pick this patch in numactl 2.0.11-2.1 level.
https:/
root@ltcgen3:
Per-node process memory usage (in MBs) for PID 53294 (qemu-system-ppc)
Node 0 Node 8 Node 252 Node 253 Node 254 Node 255 Total
------ ------ -------- -------- -------- -------- ------
Huge 0 0 0 0 0 0 0
Heap 0 28 0 0 0 0 28
Stack 0 0 0 0 0 0 0
Private 0 103727 0 0 0 0 103727
------- ------ ------ -------- -------- -------- -------- ------
Total 0 103755 0 0 0 0 103755
-------
Related branches
- Robie Basak: Approve
- Canonical Server: Pending requested
- git-ubuntu developers: Pending requested
-
Diff: 302 lines (+269/-0)5 files modifieddebian/changelog (+10/-0)
debian/patches/FTBFS-deprecated-use-readdir-3-instead.patch (+75/-0)
debian/patches/FTBFS-include-sys-sysmacros.h-for-major-minor.patch (+45/-0)
debian/patches/lp1817258-Segment-fault-when-numa-nodes-not-sequential-or-cont.patch (+136/-0)
debian/patches/series (+3/-0)
- Robie Basak: Approve
- Canonical Server: Pending requested
- git-ubuntu developers: Pending requested
-
Diff: 223 lines (+197/-0)4 files modifieddebian/changelog (+12/-0)
debian/patches/Allow-building-on-ARM-systems.patch (+47/-0)
debian/patches/lp1817258-Segment-fault-when-numa-nodes-not-sequential-or-cont.patch (+136/-0)
debian/patches/series (+2/-0)
Changed in numactl (Ubuntu): | |
status: | New → Fix Released |
Changed in ubuntu-power-systems: | |
status: | New → In Progress |
Changed in numactl (Ubuntu): | |
importance: | Undecided → High |
Changed in numactl (Ubuntu Bionic): | |
importance: | Undecided → High |
Changed in numactl (Ubuntu Cosmic): | |
importance: | Undecided → High |
Changed in ubuntu-power-systems: | |
status: | In Progress → Fix Committed |
tags: |
added: targetmilestone-inin18041 removed: targetmilestone-inin--- |
Changed in ubuntu-power-systems: | |
status: | Fix Committed → Fix Released |
Default Comment by Bridge