PowerPC: Pstore dump for powerpc is broken

Bug #1691045 reported by bugproxy on 2017-05-16
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Undecided
Canonical Kernel Team
linux (Ubuntu)
Medium
Joseph Salisbury
Zesty
Medium
Joseph Salisbury
Artful
Medium
Joseph Salisbury

Bug Description

---Problem Description---
Pstore write is broken for PowerPC. On panic, kernel doesn't write mesg to /fs/pstore/dmesg*
(Entry doesn't gets created at all)

Contact Information = Ankit <email address hidden>

---uname output---
Linux seedlp2 4.10.0-15-generic #17-Ubuntu SMP Fri Mar 24 17:50:37 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = lpar

Write to nvram failed

---Debugger---
A debugger is not configured

---Steps to Reproduce---
1. Checked kdump kernel is loaded.
2. Triggered system crash
    # echo c > /proc/sysrq-trigger

3. /sys/fs/pstore/dmesg* doesn't gets created

This issue can be reproduced after commit c950fd6f201

c950fd6f201 - pstore: Split pstore fragile flags

It introduces PSTORE_FLAGS_DMESG, which is not set for PowerPC.

Please include below commit 041939c1ec5 as this fixes this issue.

commit 041939c1ec54208b42f5cd819209173d52a29d34
Author: Ankit Kumar <email address hidden>
Date: Thu Apr 27 17:03:13 2017 +0530

    pstore: Fix flags to enable dumps on powerpc

    After commit c950fd6f201a kernel registers pstore write based on flag set.
    Pstore write for powerpc is broken as flags(PSTORE_FLAGS_DMESG) is not set for
    powerpc architecture. On panic, kernel doesn't write message to
    /fs/pstore/dmesg*(Entry doesn't gets created at all).

    This patch enables pstore write for powerpc architecture by setting
    PSTORE_FLAGS_DMESG flag.

    Fixes: c950fd6f201a ("pstore: Split pstore fragile flags")
    Cc: <email address hidden> # v4.9+
    Signed-off-by: Ankit Kumar <email address hidden>
    Signed-off-by: Kees Cook <email address hidden>

bugproxy (bugproxy) on 2017-05-16
tags: added: architecture-ppc64le bugnameltc-153411 severity-high targetmilestone-inin1704
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → linux (Ubuntu)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)

------- Comment From <email address hidden> 2017-05-16 07:07 EDT-------
Commit c950fd6f20 ("pstore: Split pstore fragile flags") is
available in kernel 4.9 and above, while commit
041939c1ec ("pstore: Fix flags to enable dumps on powerpc")
would get into 4.12, so need to be pulled into kernel version 4.9 and
above.

tags: added: kernel-da-key
Changed in linux (Ubuntu):
importance: Undecided → Medium
Changed in linux (Ubuntu Zesty):
importance: Undecided → Medium
Changed in linux (Ubuntu):
status: New → In Progress
Changed in linux (Ubuntu Zesty):
status: New → In Progress
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury)
Joseph Salisbury (jsalisbury) wrote :

I built a Zesty test kernel with a cherry pick of commit 041939c1ec54. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1691045/

Can you test this kernel and see if it resolves this bug?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-05-26 05:50 EDT-------
Issue is observed even on Ubuntu 16.04.03.

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 02:59 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 02:59 powerpc-common-nvram-6

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 02:59 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 02:59 powerpc-common-nvram-6
root@ltc-firep2:/sys/fs/pstore# tail dmesg-nvram-1
<4>[ 351.724539] LR [c0000000005dd79c] write_sysrq_trigger+0x10c/0x230
<4>[ 351.724583] Call Trace:
<4>[ 351.724603] [c000000fd79b7cf0] [c0000000005dd77c] write_sysrq_trigger+0xec/0x230 (unreliable)
<4>[ 351.724677] [c000000fd79b7d90] [c0000000003dabd4] proc_reg_write+0x84/0x120
<4>[ 351.724733] [c000000fd79b7dd0] [c000000000327520] SyS_write+0x150/0x400
<4>[ 351.724789] [c000000fd79b7e30] [c00000000000a17c] system_call+0x38/0xb4
<4>[ 351.724843] Instruction dump:
<4>[ 351.724870] 409effb8 7fc3f378 4bfff381 4bffffac 3c4c00c1 384205c0 3d42fff2 394ac530
<4>[ 351.724963] 39200001 912a0000 7c0004ac 39400000 <992a0000> 4e800020 60000000 60420000
<4>[ 351.725055] ---[ end trace 9de875846d84dfcd ]---

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 05:31 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 05:31 powerpc-common-nvram-6
root@ltc-firep2:/sys/fs/pstore# tail dmesg-nvram-1
<4>[ 351.724539] LR [c0000000005dd79c] write_sysrq_trigger+0x10c/0x230
<4>[ 351.724583] Call Trace:
<4>[ 351.724603] [c000000fd79b7cf0] [c0000000005dd77c] write_sysrq_trigger+0xec/0x230 (unreliable)
<4>[ 351.724677] [c000000fd79b7d90] [c0000000003dabd4] proc_reg_write+0x84/0x120
<4>[ 351.724733] [c000000fd79b7dd0] [c000000000327520] SyS_write+0x150/0x400
<4>[ 351.724789] [c000000fd79b7e30] [c00000000000a17c] system_call+0x38/0xb4
<4>[ 351.724843] Instruction dump:
<4>[ 351.724870] 409effb8 7fc3f378 4bfff381 4bffffac 3c4c00c1 384205c0 3d42fff2 394ac530
<4>[ 351.724963] 39200001 912a0000 7c0004ac 39400000 <992a0000> 4e800020 60000000 60420000
<4>[ 351.725055] ---[ end trace 9de875846d84dfcd ]---
root@ltc-firep2:/sys/fs/pstore#

Thanks,
Pavithra

------- Comment From <email address hidden> 2017-05-26 07:55 EDT-------
... based on previous comment .. can you please backport patch to 16.04.03 as well?

-Vasant

Joseph Salisbury (jsalisbury) wrote :

Yes, the 16.04.03 kernel is based off the 4.10 Zesty kernel, so it will get the patch. Did you happen to have a chance to test the kernel posted in comment #2?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-05-19 09:43 EDT-------
(In reply to comment #13)
> I built a Zesty test kernel with a cherry pick of commit 041939c1ec54. The
> test kernel can be downloaded from:
>
> http://kernel.ubuntu.com/~jsalisbury/lp1691045/
>
> Can you test this kernel and see if it resolves this bug?

Tested.
Working well.

~Ankit

------- Comment From <email address hidden> 2017-06-01 05:27 EDT-------
(In reply to comment #17)
> Yes, the 16.04.03 kernel is based off the 4.10 Zesty kernel, so it will get
> the patch. Did you happen to have a chance to test the kernel posted in
> comment #2?

Tested.
Working well.

~Ankit

Manoj Iyer (manjo) on 2017-06-01
tags: added: ubuntu-17.04
Changed in linux (Ubuntu Artful):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Zesty):
status: In Progress → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-zesty' to 'verification-done-zesty'. If the problem still exists, change the tag 'verification-needed-zesty' to 'verification-failed-zesty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-zesty
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-06-15 04:13 EDT-------
Above patch is present as part of :
1. Ubuntu-17.04 as well as
2. Ubuntu 16.04.2 LTS

Verified its working.
It's working as expected.

Hence moving status from verification-needed-zesty to verification-done-zesty

.
~Ankit

tags: added: verification-done-zesty
removed: verification-needed-zesty
Launchpad Janitor (janitor) wrote :
Download full text (19.7 KiB)

This bug was fixed in the package linux - 4.10.0-26.30

---------------
linux (4.10.0-26.30) zesty; urgency=low

  * linux: 4.10.0-26.30 -proposed tracker (LP: #1700528)

  * CVE-2017-1000364
    - Revert "UBUNTU: SAUCE: mm: Only expand stack if guard area is hit"
    - Revert "mm: do not collapse stack gap into THP"
    - Revert "mm: enlarge stack guard gap"
    - mm: larger stack guard gap, between vmas
    - mm: fix new crash in unmapped_area_topdown()
    - Allow stack to grow up to address space limit

linux (4.10.0-25.29) zesty; urgency=low

  * linux: 4.10.0-25.29 -proposed tracker (LP: #1699028)

  * CVE-2017-1000364
    - SAUCE: mm: Only expand stack if guard area is hit

  * CVE-2017-9074
    - ipv6: Prevent overrun when parsing v6 header options
    - ipv6: Check ip6_find_1stfragopt() return value properly.

  * [Zesty] QDF2400 ARM64 server - NMI watchdog: BUG: soft lockup - CPU#8 stuck
    for 22s! (LP: #1680549)
    - iommu/dma: Stop getting dma_32bit_pfn wrong
    - iommu/dma: Implement PCI allocation optimisation
    - iommu/dma: Convert to address-based allocation
    - iommu/dma: Clean up MSI IOVA allocation
    - iommu/dma: Plumb in the per-CPU IOVA caches
    - iommu/iova: Fix underflow bug in __alloc_and_insert_iova_range

  * Zesty update to 4.10.17 stable release (LP: #1692898)
    - xen: adjust early dom0 p2m handling to xen hypervisor behavior
    - target: Fix compare_and_write_callback handling for non GOOD status
    - target/fileio: Fix zero-length READ and WRITE handling
    - iscsi-target: Set session_fall_back_to_erl0 when forcing reinstatement
    - usb: xhci: bInterval quirk for TI TUSB73x0
    - usb: host: xhci: print correct command ring address
    - USB: serial: ftdi_sio: add device ID for Microsemi/Arrow SF2PLUS Dev Kit
    - USB: Proper handling of Race Condition when two USB class drivers try to
      call init_usb_class simultaneously
    - USB: Revert "cdc-wdm: fix "out-of-sync" due to missing notifications"
    - staging: vt6656: use off stack for in buffer USB transfers.
    - staging: vt6656: use off stack for out buffer USB transfers.
    - staging: gdm724x: gdm_mux: fix use-after-free on module unload
    - staging: wilc1000: Fix problem with wrong vif index
    - staging: comedi: jr3_pci: fix possible null pointer dereference
    - staging: comedi: jr3_pci: cope with jiffies wraparound
    - usb: misc: add missing continue in switch
    - usb: gadget: legacy gadgets are optional
    - usb: Make sure usb/phy/of gets built-in
    - usb: hub: Fix error loop seen after hub communication errors
    - usb: hub: Do not attempt to autosuspend disconnected devices
    - x86/boot: Fix BSS corruption/overwrite bug in early x86 kernel startup
    - selftests/x86/ldt_gdt_32: Work around a glibc sigaction() bug
    - x86, pmem: Fix cache flushing for iovec write < 8 bytes
    - um: Fix PTRACE_POKEUSER on x86_64
    - perf/x86: Fix Broadwell-EP DRAM RAPL events
    - KVM: x86: fix user triggerable warning in kvm_apic_accept_events()
    - KVM: arm/arm64: fix races in kvm_psci_vcpu_on
    - arm64: KVM: Fix decoding of Rt/Rt2 when trapping AArch32 CP accesses
    - block: fix blk_integrity_register to use templ...

Changed in linux (Ubuntu Zesty):
status: Fix Committed → Fix Released
Changed in linux (Ubuntu Artful):
status: Fix Committed → Fix Released
Changed in ubuntu-power-systems:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers