[Power9][Ubuntu 18.04] Can't set 8 threads per core for Power9 guest and doing it so breaks vcpu hotplug further.(kvm)

[bugproxy]

== Comment: #0 - Satheesh Rajendran <email address hidden> - 2018-02-20 02:07:45 ==
---Problem Description---
Power9 Can't set 8 threads per core for Power9 guest and doing it so breaks vcpu hotplug further.

---uname output---
4.15.0-10-generic

Machine Type = power9 boston 2.2 (pvr 004e 1202)

---Debugger---
A debugger is not configured

---Steps to Reproduce---
1. Boot a power9 guest with current vcpus=8 and threads=8 and maxvcpu 16. -- There is libvirt blocker, BUG 164807 , currently using workaround suggested as in https://bugzilla.linux.ibm.com/show_bug.cgi?id=164807#c3 to boot guest
2. Check vcpucount, vcpuinfo output, it shows 8 vcpus
3. Check vcpus inside guest, it shows only 4 vcpus and monitor info cpus says remanining 4 are halted.
4. Try vcpuhotplug, it fails inside the guest with below error
[ 108.130338] Unable to find space in cpu_present_mask for processor <NULL> with 8 thread(s)

..
   <vcpu placement='static' current='8'>16</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='ppc64le' machine='pseries-bionic'>hvm</type>
    <boot dev='hd'/>
  </os>
  <cpu>
    <topology sockets='1' cores='2' threads='8'/>
  </cpu>

...

1)# virsh start vm1

2)# virsh vcpucount vm1
maximum config 16
maximum live 16
current config 8
current live 8

3)# virsh vcpucount vm1 --guest
4 ---------------------------------------------------NOK

# virsh qemu-monitor-command vm1 --hmp --cmd "info cpus"
* CPU #0: nip=0xc0000000000b5b1c thread_id=63461
  CPU #1: nip=0xc0000000000b5b1c thread_id=63463
  CPU #2: nip=0xc0000000000b5b1c thread_id=63464
  CPU #3: nip=0xc0000000000b5b1c thread_id=63465
  CPU #4: nip=0x0000000000000100 (halted) thread_id=63466
  CPU #5: nip=0x0000000000000100 (halted) thread_id=63467
  CPU #6: nip=0x0000000000000100 (halted) thread_id=63468
  CPU #7: nip=0x0000000000000100 (halted) thread_id=63469

4) Hotplug vcpus

# virsh setvcpus vm1 16 --live

root@ltc-boston8:~# virsh vcpucount vm1 --guest
4 --------------------------------------------------------------------------NOK

guest dmesg:
[ 127.031090] Unable to find space in cpu_present_mask for processor <NULL> with 8 thread(s)
root@ub

Contact Information = <email address hidden>

Userspace tool common name: ii qemu-kvm 1:2.11+dfsg-1ubuntu1

The userspace tool has the following bit modes: both

Userspace rpm: ii qemu-kvm 1:2.11+dfsg-1ubuntu1

Userspace tool obtained from project website: na

*Additional Instructions for <email address hidden>:
-Post a private note with access information to the machine that the bug is occuring on.
-Attach ltrace and strace of userspace application.

== Comment: #1 - Satheesh Rajendran <email address hidden> - 2018-02-20 02:10:51 ==
Fix in upstream @ https://git.qemu.org/gitweb.cgi?p=qemu.git;a=history;f=target/ppc/compat.c;hb=03ee51d3548f5f553a3089f466483c1c6d5c666b

== Comment: #2 - SRIKANTH AITHAL <email address hidden> - 2018-02-20 02:21:04 ==
Please mirror this to ubuntu and request to include patch in previous comment.

[bugproxy (bugproxy)]

Default Comment by Bridge

[Christian Ehrhardt  (paelzer)]

Note to myself: this seems to be a follow on to bug 1742892 (there it was about getting it working with SMT on P9 in general).

@IBM: obviously I can't read internal bug https://bugzilla.linux.ibm.com/show_bug.cgi?id=164807#c3 - if there is something to share I'd be in general happy to know what that is about.

For the bug here:
the fix(es) you referenced are only a history on that file. All changes but one are in 2.11 anyway.
What is left is:
https://git.qemu.org/gitweb.cgi?p=qemu.git;a=commit;h=03ee51d3548f5f553a3089f466483c1c6d5c666b

This change is already in qemu 2.11.1 which I already prepare atm and is currently in regression tests.
If you want you might test from [1] which holds my current test build.

Assuming that the fix you refer will fix this and is in 2.11.1 I'll mark the changelog of the final 2.11.1 upload to fix this bug.

Let me know if you happen to pre-test from the ppa and find anything.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3159

[Launchpad Janitor (janitor)]

This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu2

---------------
qemu (1:2.11+dfsg-1ubuntu2) bionic; urgency=medium

  * d/p/ubuntu/qemu-stable-2.11.1.patch: add stable release
    - among other fixes this adds code to:
      - mitigate the Spectre/Meltdown attacks (LP: #1744882) (CVE-2017-5715)
        However, enabling this functionality requires additional configuration
        beyond just updating QEMU. Also migrations need special consideration.
        Details about that can be found at:
        https://www.qemu.org/2018/02/14/qemu-2-11-1-and-spectre-update/
      - Power9 allocation of max 8 threads per core (LP: #1750526)
  * Drop changes that are part of the upstream stable release
    - d/p/ubuntu/linux-headers-update-to-4.15-rc1.patch
    - d/p/ubuntu/linux-headers-update-4.15-rc9.patch
    - d/p/ubuntu/lp1743560-s390x-kvm-Handle-bpb-feature.patch
    - d/p/ubuntu/lp1743560-s390x-kvm-provide-stfle.81.patch
  * d/p/ubuntu/define-ubuntu-machine-types.patch: refresh to match stable update
  * d/p/ubuntu/pre-bionic-256k-ipxe-efi-roms.patch: unify to only change the
    common compat.h header and add some extra info in the patch header.

 -- Christian Ehrhardt <email address hidden> Mon, 19 Feb 2018 11:03:11 +0100

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-28 04:46 EDT-------
Tested on below qemu and found working

ii qemu-kvm 1:2.11+dfsg-1ubuntu2 ppc64el

#virsh qemu-monitor-command vm1 --hmp --cmd "info cpus"
* CPU #0: nip=0xc0000000000c009c thread_id=52837
CPU #1: nip=0xc0000000000c009c thread_id=52839
CPU #2: nip=0xc0000000000c009c thread_id=52840
CPU #3: nip=0xc0000000000c009c thread_id=52841
CPU #4: nip=0xc0000000000c009c thread_id=52842
CPU #5: nip=0xc0000000000c009c thread_id=52843
CPU #6: nip=0xc0000000000c009c thread_id=52844
CPU #7: nip=0xc0000000000c009c thread_id=52845

# virsh vcpucount vm1
maximum config 16
maximum live 16
current config 8
current live 8

# virsh setvcpus vm1 16 --live

# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 8
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0-15

# virsh setvcpus vm1 8 --live
# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0-7

Regards,
-Satheesh

------- Comment From <email address hidden> 2018-02-28 04:48 EDT-------
Closing as per previous comment