Kernel panic on shutdown

[Gustav Näslund]

Since I have updated to Ubuntu 16.10 (from 16.04) I have this problem where the shutdown takes a long time (~30s) and ends with a (at least I think it is) kernel panic with blinking harddrive- and battery status LEDs. Nothing is visible on the screen.

This didnt happen on Ubuntu 16.04 so I tried installing a 4.4 kernel and the problem disappeared.

I have then tested with various kernels from http://kernel.ubuntu.com/~kernel-ppa/mainline/ and I have found that the last good kernel is 4.7.10 and the first bad kernel is 4.8-rc1.

ProblemType: Bug
DistroRelease: Ubuntu 16.10
Package: linux-image-4.8.0-27-generic 4.8.0-27.29
ProcVersionSignature: Ubuntu 4.8.0-27.29-generic 4.8.1
Uname: Linux 4.8.0-27-generic x86_64
ApportVersion: 2.20.3-0ubuntu8
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: gustav 1998 F.... pulseaudio
 /dev/snd/controlC1: gustav 1998 F.... pulseaudio
CurrentDesktop: MATE
Date: Thu Nov 17 22:01:52 2016
HibernationDevice: RESUME=UUID=d7a5213f-0285-4708-9e97-2486aec53935
InstallationDate: Installed on 2016-09-15 (63 days ago)
InstallationMedia: Ubuntu-MATE 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
Lsusb:
 Bus 001 Device 004: ID 1bcf:28ae Sunplus Innovation Technology Inc. Laptop Integrated Webcam HD
 Bus 001 Device 003: ID 8087:0a2a Intel Corp.
 Bus 001 Device 002: ID 8087:8001 Intel Corp.
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Dell Inc. Latitude E7450
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.8.0-27-generic root=UUID=15f681e4-0a4a-4613-8810-ba7535bfa0a0 ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-4.8.0-27-generic N/A
 linux-backports-modules-4.8.0-27-generic N/A
 linux-firmware 1.161
SourcePackage: linux
UpgradeStatus: Upgraded to yakkety on 2016-10-26 (22 days ago)
dmi.bios.date: 05/17/2016
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A13
dmi.board.name: 06HN6G
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 9
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvrA13:bd05/17/2016:svnDellInc.:pnLatitudeE7450:pvr:rvnDellInc.:rn06HN6G:rvrA00:cvnDellInc.:ct9:cvr:
dmi.product.name: Latitude E7450
dmi.sys.vendor: Dell Inc.

[Brad Figg (brad-figg)]

This change was made by a bot.

[Joseph Salisbury (jsalisbury)]

I'll start a bisect between v4.7 and v4.8-rc1 and post a link to it.

Could you also see if this bug is still happening with the mainline kernel? It can be downloaded from:

http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.9-rc5/

[Joseph Salisbury (jsalisbury)]

I'll started a bisect between v4.7 and v4.8-rc1. It can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739/

Can you test this kernel after testing the v4.9-rc5 mainline kernel?

[Gustav Näslund (naslund-gustav)]

I have now tested both v4.9-rc5 and the bisect kernel (lp1642739) and both triggered the panic on shutdown.

[Joseph Salisbury (jsalisbury)]

Thanks for the update. It looks like will have to perform a kernel biscect, since 4.9-rc5 still has the bug. It will require testing about 10 - 12 test kernels.

I built the next test kernel, up to the following commit:
f7816ad0f878dacd5f0120476f9b836ccf8699ea

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

The bug is still there with this kernel.

But now I am a little confused, I thought we had already started a bisect between the good 4.7.10 and the bad 4.8-rc1 why do we need to restart the bisect from 4.9-rc5?

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
55392c4c06204c8149dc333309cf474691f1cc3c

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

That one is good, no bug!

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
27b79027bc112a63ad4004eb83c6acacae08a0de

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel was also good

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
40df3a7e586df59190547fe0e7d9928da735eada

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel is also bugfree.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
e663107fa1edda4d8a0d5b8ce704d71f8e27de43

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel has the bug!

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
e0a1e44ce0d8ea736d110b7115c5d31b0b5414c8

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This one is good, it does not have the bug.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
25474279e8fc6f15eebf89f43ac9ceedac475f57

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel did not have the bug.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
6149dffcb54b9de3a42b2ea01bc4f9d38eeff7e0

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel has the bug.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
f6bc0a168ef946344bdaf78337b59b7ee214d9b3

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel did not have the bug.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
8fc85c6ad849055c248b3d39d8c14c06dc6a7989

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel did not have the bug.

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
866ae696e26ee2b1fa0d04e67d6dafc477543fd6

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel did not have the bug

[Joseph Salisbury (jsalisbury)]

I built the next test kernel, up to the following commit:
2324d15447a9db168b1f85e3feac635b1ff8edb8

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results.

Thanks in advance

[Gustav Näslund (naslund-gustav)]

This kernel did not have the bug either.

BTW are you sure that we are testing the right kernels now? This bisect was supposed to be between v4.7 and v4.8-rc1 but the latest commit you built seems to be from 2016-06-25, which belongs to the 4.7-rc4. ?

[Joseph Salisbury (jsalisbury)]

The bisect reported the following commit as the cause of the regression:

commit 866ae696e26ee2b1fa0d04e67d6dafc477543fd6
Author: Hoan Tran <email address hidden>
Date: Thu Jun 16 14:09:38 2016 -0700

    mailbox: pcc: Add PCC request and free channel declarations

I built a Yakkety test kernel with this commit reverted. The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1642739

Can you test this kernel and see if it resolves this bug?

[Gustav Näslund (naslund-gustav)]

Unfortunately this kernel did have the bug.

However I have done some testing with the mainline kernels and I have discovered that the bug has been resolved in 4.8.15 (4.8.14 did have the bug) as well as in 4.9-rc6 (4.9-rc5 did have the bug).

[Gustav Näslund (naslund-gustav)]

I looked through the 4.8.15 changelog on https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.8.15

And I found this which is seems to be exactly my problem resolved:

commit d271b9bec2d3e7bd9a66d4b5437476f3406bbf73
Author: Rafael J. Wysocki <email address hidden>
Date: Mon Nov 21 14:25:49 2016 +0100

    Revert "ACPI: Execute _PTS before system reboot"

    commit 9713adc2a1a5488f4889c657a0c0ce0c16056d3c upstream.

    Revert commit 2c85025c75df (ACPI: Execute _PTS before system reboot)
    as it is reported to cause poweroff and reboot to hang on Dell
    Latitude E7250.

    Link: https://bugzilla.kernel.org/show_bug.cgi?id=187061
    Reported-by: Gianpaolo <email address hidden>
    Signed-off-by: Rafael J. Wysocki <email address hidden>
    Signed-off-by: Greg Kroah-Hartman <email address hidden>

[Joseph Salisbury (jsalisbury)]

That is good news. That means Yakkety will get that revert when it receives the 4.8.15 upstream updates.

[Will (lightningw9)]

This bug now affects 16.04 if you install HWE. It is present in kernel 4.8.0-36