powerpc/powernv/pci: Work around races in PCI bridge enabling
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Critical
|
Canonical Kernel Team | |||
linux (Ubuntu) |
Critical
|
Canonical Kernel Team | |||
Xenial |
Critical
|
Canonical Kernel Team | |||
Bionic |
Critical
|
Canonical Kernel Team | |||
Cosmic |
Critical
|
Canonical Kernel Team |
Bug Description
== Comment: #0 - Michael Ranweiler <email address hidden> - 2018-08-21 13:52:03 ==
+++ This bug was initially created as a clone of Bug #170766 +++
Please apply the following kernel patch:
powerpc/
The generic code is racy when multiple children of a PCI bridge try to
enable it simultaneously.
This leads to drivers trying to access a device through a
not-yet-enabled bridge, and this EEH errors under various
circumstances when using parallel driver probing.
There is work going on to fix that properly in the PCI core but it
will take some time.
x86 gets away with it because (outside of hotplug), the BIOS enables
all the bridges at boot time.
This patch does the same thing on powernv by enabling all bridges that
have child devices at boot time, thus avoiding subsequent races. It's
suitable for backporting to stable and distros, while the proper PCI
fix will probably be significantly more invasive.
Signed-off-by: Benjamin Herrenschmidt <email address hidden>
Cc: <email address hidden>
Signed-off-by: Michael Ellerman <email address hidden>
== Comment: #2 - Michael Ranweiler <email address hidden> - 2018-08-21 18:23:35 ==
This has some fuzz and also move it back from the pci macro to dev_err so we'll attach the backported patch.
tags: | added: architecture-ppc64le bugnameltc-170793 severity-critical targetmilestone-inin--- |
Changed in ubuntu: | |
assignee: | nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) |
affects: | ubuntu → kernel-package (Ubuntu) |
Changed in ubuntu-power-systems: | |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
importance: | Undecided → Critical |
tags: | added: triage-g |
affects: | kernel-package (Ubuntu) → linux (Ubuntu) |
Changed in linux (Ubuntu Bionic): | |
status: | New → In Progress |
Changed in linux (Ubuntu Cosmic): | |
status: | New → In Progress |
Changed in linux (Ubuntu Bionic): | |
importance: | Undecided → Critical |
Changed in linux (Ubuntu Cosmic): | |
importance: | Undecided → Critical |
Changed in linux (Ubuntu Bionic): | |
assignee: | nobody → Joseph Salisbury (jsalisbury) |
Changed in linux (Ubuntu Cosmic): | |
assignee: | Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury) |
Joseph Salisbury (jsalisbury) wrote : | #1 |
Changed in ubuntu-power-systems: | |
status: | New → In Progress |
Changed in linux (Ubuntu Xenial): | |
status: | New → In Progress |
importance: | Undecided → Critical |
assignee: | nobody → Joseph Salisbury (jsalisbury) |
Joseph Salisbury (jsalisbury) wrote : | #2 |
I also built a Xenial test kernel with commit db2173198b9513f
The Xenial test kernel is available from:
http://
Manoj Iyer (manjo) wrote : | #3 |
IBM, could you please verify the PPA test kernels provided by the kernel team, and report back here.
Changed in ubuntu-power-systems: | |
status: | In Progress → Incomplete |
Changed in ubuntu-power-systems: | |
status: | Incomplete → In Progress |
Changed in ubuntu-power-systems: | |
status: | In Progress → Incomplete |
Andrew Cloke (andrew-cloke) wrote : | #4 |
IBM is currently testing the test kernel from the PPA referred to in comment #2.
Changed in linux (Ubuntu): | |
status: | In Progress → Incomplete |
Changed in linux (Ubuntu Xenial): | |
status: | In Progress → Incomplete |
Changed in linux (Ubuntu Bionic): | |
status: | In Progress → Incomplete |
Changed in linux (Ubuntu Cosmic): | |
status: | In Progress → Incomplete |
Mike Ranweiler (mranweil) wrote : | #5 |
I've tried this on two systems and tried some different testing - I can't recreate the original problem that is is intermittent and can require quite a few devices, but I did not see any regressions.
Changed in ubuntu-power-systems: | |
status: | Incomplete → Triaged |
Changed in linux (Ubuntu): | |
status: | Incomplete → In Progress |
Changed in linux (Ubuntu Xenial): | |
status: | Incomplete → In Progress |
Changed in linux (Ubuntu Bionic): | |
status: | Incomplete → In Progress |
Changed in linux (Ubuntu Cosmic): | |
status: | Incomplete → In Progress |
Changed in ubuntu-power-systems: | |
status: | Triaged → In Progress |
Joseph Salisbury (jsalisbury) wrote : | #6 |
@Mike, do you feel comfortable with the testing you've done to submit an SRU request? Or would you like to perform some additional testing?
Changed in linux (Ubuntu Cosmic): | |
status: | In Progress → Fix Released |
Andrew Cloke (andrew-cloke) wrote : | #7 |
Marking as Incomplete while awaiting feedback on testing status.
Changed in ubuntu-power-systems: | |
status: | In Progress → Incomplete |
Changed in linux (Ubuntu): | |
assignee: | Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Xenial): | |
assignee: | Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Bionic): | |
assignee: | Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Cosmic): | |
assignee: | Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team) |
I built a test kernel with commit db2173198b95. The test kernel can be downloaded from: kernel. ubuntu. com/~jsalisbury /lp1788549
http://
Can you test this kernel and see if it resolves this bug?
Note about installing test kernels: unsigned .deb packages.
• If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages.
• If the test kernel is 4.15(Bionic) or newer, you need to install the linux-modules, linux-modules-extra and linux-image-
Thanks in advance!