[REGRESSION] Kernel update renders Intel NUC (i5-3427) unbootable with USB devices plugged in

[Dustin Kirkland ]

The latest kernel update in Ubuntu 14.04 LTS (3.13.0-32.57) renders our Intel NUCs unbootable, when we cold boot and have USB devices (keyboard, mouse, USB/Ethernet adapter) connected.

The 3.13.0-24.47 kernel does not exhibit this behavior.

Our particular NUCs have the i5-3427U processor, which is the D53427RKE board (DC53427HYE kit).

These reports seem to be related:
http://support.sundtek.com/index.php/topic,1600.0.html
http://en.it-usenet.org/thread/19505/17700/
---
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: ubuntu 1400 F.... pulseaudio
DistroRelease: Ubuntu 14.04
HibernationDevice: RESUME=UUID=37e4ff56-b945-4604-a94d-557bdf83e0b2
InstallationDate: Installed on 2014-07-30 (0 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.13.0-24-generic.efi.signed root=UUID=c0597da2-7d4a-4340-b85a-c7f96d8262d7 ro
ProcVersionSignature: Ubuntu 3.13.0-24.47-generic 3.13.9
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-24-generic N/A
 linux-backports-modules-3.13.0-24-generic N/A
 linux-firmware 1.127.5
Tags: trusty
Uname: Linux 3.13.0-24-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 04/25/2013
dmi.bios.vendor: Intel Corp.
dmi.bios.version: RKPPT10H.86A.0017.2013.0425.1251
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: D53427RKE
dmi.board.vendor: Intel Corporation
dmi.board.version: G87790-403
dmi.chassis.type: 3
dmi.modalias: dmi:bvnIntelCorp.:bvrRKPPT10H.86A.0017.2013.0425.1251:bd04/25/2013:svn:pn:pvr:rvnIntelCorporation:rnD53427RKE:rvrG87790-403:cvn:ct3:cvr:

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

apport information

[Dustin Kirkland  (kirkland)]

Tested an upstream kernel from jsalisbury. Seems to fix problem!

Linux OrangeBox01 3.13.0-33-generic #58~DustinTestKernelv1 SMP Wed Jul 30 17:40:39 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

[Joseph Salisbury (jsalisbury)]

A Trusty test kernel with the following commits reverted does not exhibit the bug:

7ba40e8 xhci: delete endpoints from bandwidth list before freeing whole device
8172925 usb: xhci: Prefer endpoint context dequeue pointer over stopped_trb
00bd7b9 xhci: Switch Intel Lynx Point ports to EHCI on shutdown.
c349a2e xhci: Prevent runtime pm from autosuspending during initialization

[Brad Figg (brad-figg)]

This change was made by a bot.

[Joseph Salisbury (jsalisbury)]

Built a second test kernel with only the following commits reverted:

00bd7b9 xhci: Switch Intel Lynx Point ports to EHCI on shutdown.
c349a2e xhci: Prevent runtime pm from autosuspending during initialization

It is available from:
http://kernel.ubuntu.com/~jsalisbury/kirkland/

It will probably still have the bug, I hope.

I'll bet it is:
8172925 usb: xhci: Prefer endpoint context dequeue pointer over stopped_trb

I'll also start building the next kernel with just commit 8172925 reverted.

[Joseph Salisbury (jsalisbury)]

This regression was introduced by the following commit:

commit 1f81b6d22a5980955b01e08cf27fb745dc9b686f
Author: Julius Werner <email address hidden>
Date: Fri Apr 25 19:20:13 2014 +0300

    usb: xhci: Prefer endpoint context dequeue pointer over stopped_trb

[Joseph Salisbury (jsalisbury)]

I built an upstream 3.16 test kernel with a patch from upstream. It can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/kirkland/

Can you give this kernel a test?

[Dustin Kirkland  (kirkland)]

Thanks for the test kernels, Joe. Looks like we've bisected the offending commit down to 1f81b6d22a5980955b01e08cf27fb745dc9b686f

[Joseph Salisbury (jsalisbury)]

Correct, the offending commit is 1f81b6d22a5980955b01e08cf27fb745dc9b686f. A patched has been proposed in the 3.16 kernel instead of a revert. That patch is undergoing testing now. I've also build a test kernel with this patch at:

http://kernel.ubuntu.com/~jsalisbury/kirkland/

However, that is the latest mainline kernel and not a Trusty kernel.

[Joseph Salisbury (jsalisbury)]

A patch was sent to stable. However, some of the stable kernel versions fail to build with this patch, 3.2.y and 3.13.y for example. This is because the function 'find_trb_seg' is still called by xhci_cmd_to_noop, which is removed from mainline but still exists in the stable kernels. The patch removes the definition of find_trb_seg, which is what causes the build to fail.

I built a trusty test kernel with a modified version of the upstream patch to not remove the find_trb_seg function. If this patch resolves this bug, we can ask the 3.13 upstream stable maintainer to modify the patch similarly as well as ask for an SRU to trusty with the fix.

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1350480

Can you give this latest kernel a test and see if it resolves the bug?

[Maciej Puzio (maciej-puzio)]

Joseph, thank you for pointing my attention to this bug. On my hardware (bug 1330530) the problem is not reproducible with kernel 3.13.y, so unfortunately I can neither confirm nor deny whether your test kernel fixes it. However, I would very much like to see the patch being backported to 3.2.y. i.e. Precise. I already tested the patch (modified to keep find_trb_seg) with the upstream (kernel.org) stable 3.2 kernel, and can confirm it fixes the regression.

[Brad Figg (brad-figg)]

I've installed both Trusty (3.13.0-27.50) and Utopic (3.16.0-14.20) on an Intel NUC and have not been able to reproduce this issue. I have a USB drive, USB nic and USB keyboard all plugged into the NUC.

[Brad Figg (brad-figg)]

BIOS Information
 Vendor: Intel Corp.
 Version: RKPPT10H.86A.0017.2013.0425.1251
 Release Date: 04/25/2013
 Address: 0xF0000
 Runtime Size: 64 kB
 ROM Size: 16384 kB
 Characteristics:
  PCI is supported
  BIOS is upgradeable
  BIOS shadowing is allowed
  Boot from CD is supported
  Selectable boot is supported
  EDD is supported
  5.25"/1.2 MB floppy services are supported (int 13h)
  3.5"/720 kB floppy services are supported (int 13h)
  3.5"/2.88 MB floppy services are supported (int 13h)
  Print screen service is supported (int 5h)
  Serial services are supported (int 14h)
  Printer services are supported (int 17h)
  ACPI is supported
  USB legacy is supported
  BIOS boot specification is supported
  Targeted content distribution is supported
  UEFI is supported
 BIOS Revision: 4.6

Base Board Information
 Manufacturer: Intel Corporation
 Product Name: D53427RKE
 Version: G87971-402
 Serial Number: GERK32900730
 Asset Tag: To be filled by O.E.M.
 Features:
  Board is a hosting board
  Board is replaceable
 Location In Chassis: To be filled by O.E.M.
 Chassis Handle: 0x0003
 Type: Motherboard
 Contained Object Handles: 0

[Brad Figg (brad-figg)]

I just reinstalled via MAAS. Once up I "sudo apt-get update; sudo apt-get dist-upgrade" and then rebooted. I'm back up.

Linux nuc1 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

[Joseph Salisbury (jsalisbury)]

A possible fix for this is going into the 3.13 upstream stable kernel:
https://lists.ubuntu.com/archives/kernel-team/2014-September/048880.html

Is this still an issue on any NUCs? If so, possible test 3.13.11.8 when it is available and see if it resolves this bug.

[Steve Langasek (vorlon)]

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release