PowerPC: Pstore dump for powerpc is broken

Bug #1691045 reported by bugproxy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Fix Released
Undecided
Canonical Kernel Team
linux (Ubuntu)
Fix Released
Medium
Joseph Salisbury
Zesty
Fix Released
Medium
Joseph Salisbury
Artful
Fix Released
Medium
Joseph Salisbury

Bug Description

---Problem Description---
Pstore write is broken for PowerPC. On panic, kernel doesn't write mesg to /fs/pstore/dmesg*
(Entry doesn't gets created at all)

Contact Information = Ankit <email address hidden>

---uname output---
Linux seedlp2 4.10.0-15-generic #17-Ubuntu SMP Fri Mar 24 17:50:37 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = lpar

Write to nvram failed

---Debugger---
A debugger is not configured

---Steps to Reproduce---
1. Checked kdump kernel is loaded.
2. Triggered system crash
    # echo c > /proc/sysrq-trigger

3. /sys/fs/pstore/dmesg* doesn't gets created

This issue can be reproduced after commit c950fd6f201

c950fd6f201 - pstore: Split pstore fragile flags

It introduces PSTORE_FLAGS_DMESG, which is not set for PowerPC.

Please include below commit 041939c1ec5 as this fixes this issue.

commit 041939c1ec54208b42f5cd819209173d52a29d34
Author: Ankit Kumar <email address hidden>
Date: Thu Apr 27 17:03:13 2017 +0530

    pstore: Fix flags to enable dumps on powerpc

    After commit c950fd6f201a kernel registers pstore write based on flag set.
    Pstore write for powerpc is broken as flags(PSTORE_FLAGS_DMESG) is not set for
    powerpc architecture. On panic, kernel doesn't write message to
    /fs/pstore/dmesg*(Entry doesn't gets created at all).

    This patch enables pstore write for powerpc architecture by setting
    PSTORE_FLAGS_DMESG flag.

    Fixes: c950fd6f201a ("pstore: Split pstore fragile flags")
    Cc: <email address hidden> # v4.9+
    Signed-off-by: Ankit Kumar <email address hidden>
    Signed-off-by: Kees Cook <email address hidden>

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-153411 severity-high targetmilestone-inin1704
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2017-05-16 07:07 EDT-------
Commit c950fd6f20 ("pstore: Split pstore fragile flags") is
available in kernel 4.9 and above, while commit
041939c1ec ("pstore: Fix flags to enable dumps on powerpc")
would get into 4.12, so need to be pulled into kernel version 4.9 and
above.

tags: added: kernel-da-key
Changed in linux (Ubuntu):
importance: Undecided → Medium
Changed in linux (Ubuntu Zesty):
importance: Undecided → Medium
Changed in linux (Ubuntu):
status: New → In Progress
Changed in linux (Ubuntu Zesty):
status: New → In Progress
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury)
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

I built a Zesty test kernel with a cherry pick of commit 041939c1ec54. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1691045/

Can you test this kernel and see if it resolves this bug?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-05-26 05:50 EDT-------
Issue is observed even on Ubuntu 16.04.03.

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 02:59 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 02:59 powerpc-common-nvram-6

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 02:59 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 02:59 powerpc-common-nvram-6
root@ltc-firep2:/sys/fs/pstore# tail dmesg-nvram-1
<4>[ 351.724539] LR [c0000000005dd79c] write_sysrq_trigger+0x10c/0x230
<4>[ 351.724583] Call Trace:
<4>[ 351.724603] [c000000fd79b7cf0] [c0000000005dd77c] write_sysrq_trigger+0xec/0x230 (unreliable)
<4>[ 351.724677] [c000000fd79b7d90] [c0000000003dabd4] proc_reg_write+0x84/0x120
<4>[ 351.724733] [c000000fd79b7dd0] [c000000000327520] SyS_write+0x150/0x400
<4>[ 351.724789] [c000000fd79b7e30] [c00000000000a17c] system_call+0x38/0xb4
<4>[ 351.724843] Instruction dump:
<4>[ 351.724870] 409effb8 7fc3f378 4bfff381 4bffffac 3c4c00c1 384205c0 3d42fff2 394ac530
<4>[ 351.724963] 39200001 912a0000 7c0004ac 39400000 <992a0000> 4e800020 60000000 60420000
<4>[ 351.725055] ---[ end trace 9de875846d84dfcd ]---

root@ltc-firep2:/sys/fs/pstore# ls -lrt
total 0
-r--r--r-- 1 root root 3909 May 5 04:21 dmesg-nvram-1
-r--r--r-- 1 root root 4080 May 26 05:31 powerpc-opal-nvram-8
-r--r--r-- 1 root root 65520 May 26 05:31 powerpc-common-nvram-6
root@ltc-firep2:/sys/fs/pstore# tail dmesg-nvram-1
<4>[ 351.724539] LR [c0000000005dd79c] write_sysrq_trigger+0x10c/0x230
<4>[ 351.724583] Call Trace:
<4>[ 351.724603] [c000000fd79b7cf0] [c0000000005dd77c] write_sysrq_trigger+0xec/0x230 (unreliable)
<4>[ 351.724677] [c000000fd79b7d90] [c0000000003dabd4] proc_reg_write+0x84/0x120
<4>[ 351.724733] [c000000fd79b7dd0] [c000000000327520] SyS_write+0x150/0x400
<4>[ 351.724789] [c000000fd79b7e30] [c00000000000a17c] system_call+0x38/0xb4
<4>[ 351.724843] Instruction dump:
<4>[ 351.724870] 409effb8 7fc3f378 4bfff381 4bffffac 3c4c00c1 384205c0 3d42fff2 394ac530
<4>[ 351.724963] 39200001 912a0000 7c0004ac 39400000 <992a0000> 4e800020 60000000 60420000
<4>[ 351.725055] ---[ end trace 9de875846d84dfcd ]---
root@ltc-firep2:/sys/fs/pstore#

Thanks,
Pavithra

------- Comment From <email address hidden> 2017-05-26 07:55 EDT-------
... based on previous comment .. can you please backport patch to 16.04.03 as well?

-Vasant

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

Yes, the 16.04.03 kernel is based off the 4.10 Zesty kernel, so it will get the patch. Did you happen to have a chance to test the kernel posted in comment #2?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-05-19 09:43 EDT-------
(In reply to comment #13)
> I built a Zesty test kernel with a cherry pick of commit 041939c1ec54. The
> test kernel can be downloaded from:
>
> http://kernel.ubuntu.com/~jsalisbury/lp1691045/
>
> Can you test this kernel and see if it resolves this bug?

Tested.
Working well.

~Ankit

------- Comment From <email address hidden> 2017-06-01 05:27 EDT-------
(In reply to comment #17)
> Yes, the 16.04.03 kernel is based off the 4.10 Zesty kernel, so it will get
> the patch. Did you happen to have a chance to test the kernel posted in
> comment #2?

Tested.
Working well.

~Ankit

Manoj Iyer (manjo)
tags: added: ubuntu-17.04
Changed in linux (Ubuntu Artful):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Zesty):
status: In Progress → Fix Committed
Revision history for this message
Kleber Sacilotto de Souza (kleber-souza) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-zesty' to 'verification-done-zesty'. If the problem still exists, change the tag 'verification-needed-zesty' to 'verification-failed-zesty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-zesty
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2017-06-15 04:13 EDT-------
Above patch is present as part of :
1. Ubuntu-17.04 as well as
2. Ubuntu 16.04.2 LTS

Verified its working.
It's working as expected.

Hence moving status from verification-needed-zesty to verification-done-zesty

.
~Ankit

tags: added: verification-done-zesty
removed: verification-needed-zesty
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (19.7 KiB)

This bug was fixed in the package linux - 4.10.0-26.30

---------------
linux (4.10.0-26.30) zesty; urgency=low

  * linux: 4.10.0-26.30 -proposed tracker (LP: #1700528)

  * CVE-2017-1000364
    - Revert "UBUNTU: SAUCE: mm: Only expand stack if guard area is hit"
    - Revert "mm: do not collapse stack gap into THP"
    - Revert "mm: enlarge stack guard gap"
    - mm: larger stack guard gap, between vmas
    - mm: fix new crash in unmapped_area_topdown()
    - Allow stack to grow up to address space limit

linux (4.10.0-25.29) zesty; urgency=low

  * linux: 4.10.0-25.29 -proposed tracker (LP: #1699028)

  * CVE-2017-1000364
    - SAUCE: mm: Only expand stack if guard area is hit

  * CVE-2017-9074
    - ipv6: Prevent overrun when parsing v6 header options
    - ipv6: Check ip6_find_1stfragopt() return value properly.

  * [Zesty] QDF2400 ARM64 server - NMI watchdog: BUG: soft lockup - CPU#8 stuck
    for 22s! (LP: #1680549)
    - iommu/dma: Stop getting dma_32bit_pfn wrong
    - iommu/dma: Implement PCI allocation optimisation
    - iommu/dma: Convert to address-based allocation
    - iommu/dma: Clean up MSI IOVA allocation
    - iommu/dma: Plumb in the per-CPU IOVA caches
    - iommu/iova: Fix underflow bug in __alloc_and_insert_iova_range

  * Zesty update to 4.10.17 stable release (LP: #1692898)
    - xen: adjust early dom0 p2m handling to xen hypervisor behavior
    - target: Fix compare_and_write_callback handling for non GOOD status
    - target/fileio: Fix zero-length READ and WRITE handling
    - iscsi-target: Set session_fall_back_to_erl0 when forcing reinstatement
    - usb: xhci: bInterval quirk for TI TUSB73x0
    - usb: host: xhci: print correct command ring address
    - USB: serial: ftdi_sio: add device ID for Microsemi/Arrow SF2PLUS Dev Kit
    - USB: Proper handling of Race Condition when two USB class drivers try to
      call init_usb_class simultaneously
    - USB: Revert "cdc-wdm: fix "out-of-sync" due to missing notifications"
    - staging: vt6656: use off stack for in buffer USB transfers.
    - staging: vt6656: use off stack for out buffer USB transfers.
    - staging: gdm724x: gdm_mux: fix use-after-free on module unload
    - staging: wilc1000: Fix problem with wrong vif index
    - staging: comedi: jr3_pci: fix possible null pointer dereference
    - staging: comedi: jr3_pci: cope with jiffies wraparound
    - usb: misc: add missing continue in switch
    - usb: gadget: legacy gadgets are optional
    - usb: Make sure usb/phy/of gets built-in
    - usb: hub: Fix error loop seen after hub communication errors
    - usb: hub: Do not attempt to autosuspend disconnected devices
    - x86/boot: Fix BSS corruption/overwrite bug in early x86 kernel startup
    - selftests/x86/ldt_gdt_32: Work around a glibc sigaction() bug
    - x86, pmem: Fix cache flushing for iovec write < 8 bytes
    - um: Fix PTRACE_POKEUSER on x86_64
    - perf/x86: Fix Broadwell-EP DRAM RAPL events
    - KVM: x86: fix user triggerable warning in kvm_apic_accept_events()
    - KVM: arm/arm64: fix races in kvm_psci_vcpu_on
    - arm64: KVM: Fix decoding of Rt/Rt2 when trapping AArch32 CP accesses
    - block: fix blk_integrity_register to use templ...

Changed in linux (Ubuntu Zesty):
status: Fix Committed → Fix Released
Changed in linux (Ubuntu Artful):
status: Fix Committed → Fix Released
Changed in ubuntu-power-systems:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.