[18.04][LTCTest][OPAL][OP930] IMC PMU (null) Register failed

[bugproxy]

== Comment: #0 - PAVAMAN SUBRAMANIYAM <email address hidden> - 2019-04-10 00:15:32 ==
Install a P9 Open Power Hardware with the latest OP930 Firmware images provided in the following link:
http://pfd.austin.ibm.com/releasenotes/openpower9/OP930/OP930_1913A/OP930_1913B_RelNote_Main.html

root@witherspoon:~# cat /etc/os-release
ID="openbmc-phosphor"
NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)"
VERSION="ibm-v2.3"
VERSION_ID="ibm-v2.3-476-g2d622cb-r32-0-g9973ab0"
PRETTY_NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro) ibm-v2.3"
BUILD_ID="ibm-v2.3-476-g2d622cb-r32"
root@witherspoon:~# cat /var/lib/phosphor-software-manager/pnor/ro/VERSION
 IBM-witherspoon-OP9-v2.2-9.71
        op-build-v2.2-58-gf60d718
        buildroot-2019.02-3-g28a7eab
        skiboot-v6.3-rc1
        hostboot-30a512e-pf453f1d
        occ-58e422d
        linux-4.19.30-openpower1-pa764076
        petitboot-v1.10.2
        machine-xml-e3e9aef
        hostboot-binaries-hw021419a.930
        capp-ucode-p9-dd2-v4
        sbe-1410677
        hcode-hw032719a.940

Then seeing the following messages in dmesg of Linux HOST OS and also when we reboot the machine in the SOL console.

[ 1.731663] nest_phb0_imc performance monitor hardware support registered
[ 1.731704] nest_phb1_imc performance monitor hardware support registered
[ 1.731746] nest_phb2_imc performance monitor hardware support registered
[ 1.731788] nest_phb3_imc performance monitor hardware support registered
[ 1.731829] nest_phb4_imc performance monitor hardware support registered
[ 1.731870] nest_phb5_imc performance monitor hardware support registered
[ 1.731944] nest_powerbus0_imc performance monitor hardware support registered
[ 1.734458] thread_imc performance monitor hardware support registered
[ 1.734460] IMC Unknown Device type
[ 1.734462] IMC PMU (null) Register failed
[ 1.734558] nest_xlink0_imc performance monitor hardware support registered
[ 1.734614] nest_xlink1_imc performance monitor hardware support registered
[ 1.734670] nest_xlink2_imc performance monitor hardware support registered
[ 1.747043] Initialise system trusted keyrings
[ 1.747054] Key type blacklist registered
[ 1.747170] workingset: timestamp_bits=38 max_order=23 bucket_order=0

== Comment: #2 - PAVAMAN SUBRAMANIYAM <email address hidden> - 2019-04-10 00:30:47 ==
So should this bug be moved to kernel to get the fixes be made available to Ubuntu 18.04 release or should we keep this bug opened for skiboot under OPAL itself, since we are already in the upstream level of skiboot installed on the system.

== Comment: #5 - Application Cdeadmin <email address hidden> - 2019-04-11 15:15:13 ==
==== State: Open by: jayeshp on 11 April 2019 15:05:22 ====

#=#=# 2019-04-11 15:05:21 (CDT) #=#=#
New Fix_Potential = [P910.40W]
#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#

== Comment: #7 - Application Cdeadmin <email address hidden> - 2019-04-16 06:12:44 ==
cde00 (<email address hidden>) added native attachment /tmp/AIXOS11139536/0001-Return-for-invalid-domain.patch on 2019-04-16 06:12:39

== Comment: #8 - Deborah L. McLemore <email address hidden> - 2019-04-20 09:58:13 ==

== Comment: #9 - Application Cdeadmin <email address hidden> - 2019-04-26 08:25:05 ==
==== State: Open by: csburns on 26 April 2019 08:15:35 ====

The patch in comment #7 was included in a test pnor driver - op999.1999.20190423t - that was provided to ISST to look at several skiboot pending changes. This defect was also seen in System Mfg Test - so this patch should be merged for OP930 GA.

== Comment: #10 - VASANT HEGDE <email address hidden> - 2019-05-14 23:02:35 ==
Anju,

Where are we with this patch? Did you push changes to upstream?

Also is this really a ship issue?

-Vasant

== Comment: #15 - Klaus Heinrich Kiwi <email address hidden> - 2020-03-25 06:27:01 ==
(In reply to comment #13)
> Patch is now upstream.
> https://git.kernel.org/powerpc/c/b59bd3527fe3c1939340df558d7f9d56
>
> Thanks,
> Anju

== Comment: #16 - Klaus Heinrich Kiwi <email address hidden> - 2020-03-25 06:33:24 ==
(In reply to comment #14)
> (In reply to comment #13)
> > Patch is now upstream.
> > https://git.kernel.org/powerpc/c/b59bd3527fe3c1939340df558d7f9d56
> >
> > Thanks,
> > Anju
>
> What next? will you be sending fix to distros?
>
> -Vasant

[Frank Heimes (fheimes)]

Since there is the tag "targetmilestone-inin2004" set, I assume this bug was opened against focal / 20.04.

Doing a look-up of b59bd3527fe3 "powerpc/powernv: Return for invalid IMC domain" in the focal master tree (that correlates to the kernel that is currently in focal's release pocket and is with that the current default kernel) shows that this specific patch/commit is already in:

$ git clone https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal
...
$ git log --oneline --grep "powerpc/powernv: Return for invalid IMC domain"
b59bd3527fe3 powerpc/powernv: Return for invalid IMC domain
$ git tag --contains b59bd3527fe3
Ubuntu-5.4-5.4.0-10.13
Ubuntu-5.4-5.4.0-11.14
Ubuntu-5.4-5.4.0-12.15
Ubuntu-5.4-5.4.0-13.16
Ubuntu-5.4-5.4.0-14.17
Ubuntu-5.4.0-15.18
Ubuntu-5.4.0-16.19
Ubuntu-5.4.0-17.20
Ubuntu-5.4.0-17.21
Ubuntu-5.4.0-18.22
Ubuntu-5.4.0-19.23
Ubuntu-5.4.0-20.24
Ubuntu-5.4.0-8.11
Ubuntu-5.4.0-9.12
v5.4

So I'm wondering which kernel version was used in your case?
Please can you share the output of 'uname -a'?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-03-26 08:09 EDT-------
Here is the kernel version on which we see the original issue.

root@ltc-wspoon11:~# uname -a
Linux ltc-wspoon11 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:08:26 UTC 2020 ppc64le ppc64le ppc64le GNU/Linux
root@ltc-wspoon11:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

[Andrew Cloke (andrew-cloke)]

Thanks. As this issue is seen with the 4.15 Bionic (18.04) kernel, I'll update the tags and title accordingly. From Frank's comment above, I think we can assume that this is fixed in Focal (20.04).

[Andrew Cloke (andrew-cloke)]

From comment #2, this issue is related to 18.04, not 20.04. Please add a comment with rationale when updating tags.
Thanks.

[Frank Heimes (fheimes)]

I just wanted to start to work on the SRU of "powerpc/powernv: Return for invalid IMC domain" to bionic. First of all I looked up when the patch got upstream accepted and when it landed in linux (master):
$ git tag --contains b59bd3527fe3
v5.2-rc3

With that it must be already in Eoan, since Eoan was shipped with a kernel 5.3 (notice that we always need to SRU from newest to oldest - incl. all Ubuntu releases that are in service).
Hence looking it up in Eoan (master):
$ git tag --contains b59bd3527fe3
Ubuntu-5.2.0-8.9
...
Ubuntu-5.3.0-45.37
Means it is already in Eoan - so we are good there.

Now checking bionic (master):
$ git log --oneline --grep "powerpc/powernv: Return for invalid IMC domain"
062b1248da71 powerpc/powernv: Return for invalid IMC domain
$ git tag --contains 062b1248da71
Ubuntu-4.15.0-59.66
...
Ubuntu-4.15.0-94.95
So it is already in bionic, too!

How did it found it's way into bionic:
$ git show 062b1248da71 | grep BugLink
    BugLink: https://bugs.launchpad.net/bugs/1838824
$ lp-list-bugs 1838824
Bug 1838824: Bionic update: upstream stable patchset 2019-08-02
  linux (Ubuntu): Fix Released
  linux (Ubuntu Bionic): Fix Released

In summary the commit/patch "powerpc/powernv: Return for invalid IMC domain" landed in bionics kernel 'Ubuntu-4.15.0-59.66' via the bionic upstream stable patchset from 2019-08-02.

If this patch/commit is the right one, then there seems to be nothing todo here.

[Frank Heimes (fheimes)]

As discussed in call, I'm closing this ticket as Fix Released, since the commit already landed in bionic via an upstream stable patch set.
In case more is needed on top, a new ticket can be opened any time.