incorrect hypervisor and virtualization type reported in compat mode guest

[bugproxy]

[IMPACT]

In xenial lscpu prints the wrong "Hypervisor vendor" and "Virtualization type" on PowerVM or KVM systems. Incorrect hypervisor and virtualization type reported in ubuntu 16.04.04 guest running in P8compat mode on P9 boston-LC.

[TEST]

Curent output:

ubuntu@P8lpar3:~$ dpkg -l "*util-linux*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii util-linux 2.31.1-0.4ub ppc64el miscellaneous system utilities
un util-linux-loc <none> <none> (no description available)

ubuntu@P8lpar3:~$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 16
NUMA node(s): 2
Model: 2.1 (pvr 004b 0201)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: horizontal
Virtualization type: full
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s):
NUMA node4 CPU(s): 0-127
ubuntu@P8lpar3:~$

Expected Output:

$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 16
NUMA node(s): 2
Model: 2.1 (pvr 004b 0201)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: pHyp
Virtualization type: para
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s):
NUMA node4 CPU(s): 0-127

[Potential Regression]
The fix changes the logic to how lscpu-dmi returns from read_hypervisor_dmi() this could introduce potential regression in platforms that has incorrect DMI information.

[Other Info]
---uname output---
Linux guest 4.15.0-13-generic #14~16.04.1-Ubuntu SMP Sat Mar 17 03:03:53 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = boston-LC

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 Incorrect hypervisor and virtualization type reported in ubuntu 16.04.04 guest running in P8compat mode on P9 boston-LC:

root@guest:/tmp# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER8 (architected), altivec supported
>> Hypervisor vendor: horizontal
>> Virtualization type: full
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0,1

Stack trace output:
 no

Oops output:
 no

We test what is coming along with distro. If you are not able to see issue with : https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.5 .. can we get this included in 16.04.x train ?

[bugproxy (bugproxy)]

Default Comment by Bridge

[Steve Langasek (vorlon)]

There seems to be no reason for https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.5 to fix this bug, since that is a targeted bugfix; and I understand your bug report to be asking if it's reproducible with this version, rather than telling that it is not.

What are the values you expect in the output of lscpu for the "Hypervisor vendor" and "Virtualization type" fields that you highlight? In what upstream version of util-linux is this known to be fixed?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-04-18 01:24 EDT-------
(In reply to comment #13)
> There seems to be no reason for
> https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.5 to fix
> this bug, since that is a targeted bugfix; and I understand your bug report
> to be asking if it's reproducible with this version, rather than telling
> that it is not.
>
> What are the values you expect in the output of lscpu for the "Hypervisor
> vendor" and "Virtualization type" fields that you highlight? In what
> upstream version of util-linux is this known to be fixed?

We have not seen this issue in https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.5 version.
So we could like to know if you could include this version in 16.04.04 Ubuntu version..

The expected value is

Hypervisor vendor: KVM
Virtualization type: para

root@guest:~/util-linux# ./lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0,1

We downloaded the below upstream code and did not see the issue.

https://github.com/Distrotech/util-linux

[Steve Langasek (vorlon)]

it is not at all clear, based on the source changes included, why https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.5 would fix this issue.

2.27.1-6ubuntu3.5 is also currently blocked from being released to xenial-updates because the update failed verification. https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1732865

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-06-27 01:45 EDT-------
(In reply to comment #16)
> Srikanth, Seeteena,
>
> Canonical has asked for an update to this bug. Could you provide one?
> Specifically, their question is... "please can you provide some more details
> why this is fixed with this util-linux version?" Thanks.

Barry, I am not able to see any commit that went with util-linux_2.27.1-6ubuntu3.5 version. but from the testing perspective we are able to recreate the issue with the util-linux package that comes with 16.04.04.
Later we checked with upstream/util-linux_2.27.1-6ubuntu3.5 level and not able to see the issue. Hence we mirror this bug to canonical to pick or use this package for 16.04.04.

http://launchpadlibrarian.net/356140271/util-linux_2.27.1-6ubuntu3.5_source.changes

[Manoj Iyer (manjo)]

IBM, Xenial-updates is currently on version (https://packages.ubuntu.com/xenial-updates/util-linux) util-linux (2.27.1-6ubuntu3.6) could you please retest and confirm that this is now fixed ?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-12-18 01:25 EDT-------
(In reply to comment #18)
> IBM, Xenial-updates is currently on version
> (https://packages.ubuntu.com/xenial-updates/util-linux) util-linux
> (2.27.1-6ubuntu3.6) could you please retest and confirm that this is now
> fixed ?

No. Issue still present:

root@guest:~# dpkg -l | grep util-linux
ii util-linux 2.27.1-6ubuntu3.6 ppc64el miscellaneous system utilities
root@guest:~# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 4
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 1
Model: 2.0 (pvr 004d 0200)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: horizontal
Virtualization type: full
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0-63

[Dimitri John Ledkov (xnox)]

I'm confused reading this bug report. As the comments repeatedly mention identical version numbers, stating that both the issue is present and not present, requesting to pick up the version that is already in xenial, and later stating again that things are not fixed.

Clearly v2.27.1-6ubuntu3.6 is still affected and is not the one that is needed.

Do you have upstream git commit ids that resolve the issue for you?

What does "Later we checked with upstream/util-linux_2.27.1-6ubuntu3.5 level and not able to see the issue." mean? i do not understand the meaning of this sentence. Which upstream git commit, or git tag, or released version number appear to work correctly?

[Dimitri John Ledkov (xnox)]

Upstream for util-linux is at https://github.com/karelzak/util-linux

Has this been reported as an issue there before, and now fixed?
Can you find the relevant versions that fix this in that repository?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-12-19 23:26 EDT-------
Upstream testing with util-linux [@ https://github.com/karelzak/util-linux]

1. Tested version: util-linux 2.26.2

Result:

root@guest:/home/srik/util-linux# ./lscpu -V
lt-lscpu from util-linux 2.26.2
root@guest:/home/srik/util-linux# ./lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 4
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 1
Model: IBM pSeries (emulated by qemu)
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0-63

2. Tested version: util-linux 2.27.1

Result:

root@guest:/home/srik/util-linux# ./lscpu -V
lt-lscpu from util-linux 2.27.1
root@guest:/home/srik/util-linux# ./lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 4
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 1
Model: IBM pSeries (emulated by qemu)
> Hypervisor vendor: KVM --> correct
> Virtualization type: para --> correct
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0-63

On Ubuntu 16.04.5 :

#lscpu -V
lscpu from util-linux 2.27.1

root@guest:~# dpkg -l | grep util-linux
ii util-linux 2.27.1-6ubuntu3.6 ppc64el miscellaneous system utilities

Result:
root@guest:~# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 4
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 1
Model: 2.0 (pvr 004d 0200)
Model name: POWER8 (architected), altivec supported
> Hypervisor vendor: horizontal --> not ok
> Virtualization type: full --> not ok
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0-63

I hope its clear now.

------- Comment From <email address hidden> 2018-12-19 23:27 EDT-------
I am on:

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial

[Dimitri John Ledkov (xnox)]

This is very clear now, thank you!

[Dimitri John Ledkov (xnox)]

Ok. so in xenial, on top of upstream release we cherrypick a few lscpu "fixes" for various arches, including powerpc from, what at the time was development. Clearly we didn't cherrypick enough of things, hence the lscpu regression. By 2.31 as used in bionic, things are all fine again.

So there must be patches in upstream util-linux between 2.27 and 2.31 which we do not have, but need to fix this.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-03-15 00:39 EDT-------
Any update on this bug ?

[Frank Heimes (fheimes)]

IBM, please can you provide us with a list of patches (probably from the delta between 2.27 and 2.31) that needs to be applied on top of 2.27 to get the situation solved?
(For already released Ubuntu versions we cannot simply bump the version to 2.31 - the patches need to be cherry-picked ...)

[Manoj Iyer (manjo)]

Please review and consider this debdiff to Xenial. It fixes the issue reported here.

== Before Patch ==

ubuntu@P8lpar4:~$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 16
NUMA node(s): 2
Model: 2.1 (pvr 004b 0201)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: horizontal
Virtualization type: full
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s):
NUMA node2 CPU(s): 0-127
ubuntu@P8lpar4:~$

== After Patch ==

ubuntu@P8lpar4:~$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 16
NUMA node(s): 2
Model: 2.1 (pvr 004b 0201)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: pHyp
Virtualization type: para
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s):
NUMA node2 CPU(s): 0-127
ubuntu@P8lpar4:~$

[Manoj Iyer (manjo)]

A test package is available in PPA:

https://launchpad.net/~manjo/+archive/ubuntu/lp1764628/

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "Fix to identify virtualization type correctly in Xenial on PowerPC" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Dimitri John Ledkov (xnox)]

In unapproved

[Manoj Iyer (manjo)]

xnox, would you please elaborate what "in unapproved" means ?

[Steve Langasek (vorlon)]

It's in the xenial-proposed queue.

Marking fixed for trunk.

[Brian Murray (brian-murray)]

Hello bugproxy, or anyone else affected,

Accepted util-linux into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/util-linux/2.27.1-6ubuntu3.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Manoj Iyer (manjo)]

ubuntu@P8lpar2:~$ apt policy util-linux
util-linux:
  Installed: 2.27.1-6ubuntu3.7
  Candidate: 2.27.1-6ubuntu3.7
  Version table:
 *** 2.27.1-6ubuntu3.7 500
        500 http://ports.ubuntu.com/ubuntu-ports xenial-proposed/main ppc64el Packages

ubuntu@P8lpar2:~$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 256
On-line CPU(s) list: 0-255
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 32
NUMA node(s): 2
Model: 2.1 (pvr 004b 0201)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: pHyp
Virtualization type: para
L1d cache: 64K
L1i cache: 32K
NUMA node0 CPU(s): 0-255
NUMA node1 CPU(s):
ubuntu@P8lpar2:~$

[Launchpad Janitor (janitor)]

This bug was fixed in the package util-linux - 2.27.1-6ubuntu3.7

---------------
util-linux (2.27.1-6ubuntu3.7) xenial; urgency=medium

  * d/patches/lscpu-cleanup-DMI-detection-return-codes.patch:
    Cherry pick upstream patch to fix correctly identifying
    virtualization type.
    LP: #1764628

 -- Manoj Iyer <email address hidden> Tue, 02 Apr 2019 17:20:29 -0500

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for util-linux has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-06-20 07:09 EDT-------
Verified with `util-linux 2.31.1-0.4ubuntu3.3`

Issue is fixed.

Inside compat guest:

# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 4
Core(s) per socket: 2
Socket(s): 2
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER8 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0-15