intel-microcode 3.20180312.0 causes lockup at login screen

[Erich E. Hoover]

[Impact]

 * Some systems experience kernel lockups after updating to the latest intel-microcode
   package or when receiving updated microcode from a BIOS update.

 * In many cases, the lockups occur before users can reach the login screen which makes
   it very difficult to debug/workaround.

[Test Case]

 * The most reliable test case currently known is to install the sssd package. Lockups
   may occur during package installation (disable IBPB by writing 0 to
   /proc/sys/kernel/ibpb_enabled to prevent this from happening). A lockup will most
   likely occur just after booting the system up as the lock screen is displayed.

[Regression Potential]

 * The fix is in the task switching code of the kernel so complexity of the change is
   relatively high.

[Original Report]

I don't know if this is a problem with the kernel or the microcode, but we have a significant number of computers in our organization (on both 16.04 and 17.10) that fail if they have both updated. Booting with either linux-image-4.13.0-36-generic or intel-microcode 3.20180108.0+really20170707ubuntu17.10.1 allows all these computers to boot.

## Workaround ##
1. Boot the system with the dis_ucode_ldr kernel boot parameter to temporary avoid the problem:
   https://wiki.ubuntu.com/Kernel/KernelBootParameters
2. Install the previous version of package from
   https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+build/14261530/+files/intel-microcode_3.20180108.0+really20170707ubuntu16.04.1_amd64.deb
3. (Optional) Hold the package so that it won't be upgraded accidentally
   sudo apt-mark hold intel-microcode

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1759920

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tyler Hicks (tyhicks)]

Thanks for the report, Erich!

Does the lockup happen as soon as you reach the login screen or does it happen once you put in your password and hit enter?

Do you happen to use NVIDIA graphics drivers on the system that's locking up?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jeremy Gebben (jgebben)]

ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
DistroRelease: Ubuntu 17.10
NonfreeKernelModules: nvidia_uvm nvidia_drm nvidia_modeset nvidia
Package: linux
PackageArchitecture: amd64
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
 TERM=screen-256color
 XDG_RUNTIME_DIR=<set>
 PATH=(custom, no user)
ProcVersionSignature: Ubuntu 4.13.0-36.40-generic 4.13.13
Tags: artful
Uname: Linux 4.13.0-36-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: Domain Users adm cdrom dialout dip lpadmin plugdev sambashare sbuild sudo wireshark
_MarkForUpload: True

[Jeremy Gebben (jgebben)]

apport information

[Jeremy Gebben (jgebben)]

apport information

[Jeremy Gebben (jgebben)]

apport information

[Tyler Hicks (tyhicks)]

Erich, could you include the output of 'iucode-tool --scan-system'?

[Erich E. Hoover (ehoover)]

> Does the lockup happen as soon as you reach the login screen or does it happen once you put in your password and hit enter?

It sometimes happens as soon as the login screen shows up, sometimes it happens after you hit enter, and roughly 1:8 tries it will make it all the way to the desktop.

> Do you happen to use NVIDIA graphics drivers on the system that's locking up?

I believe that all of our systems are running NVIDIA drivers, but we have replicated it on a system not running the NVIDIA driver.

> Erich, could you include the output of 'iucode-tool --scan-system'?

Most of them report:
iucode-tool: system has processor(s) with signature 0x000906e9
We have one that reports:
iucode-tool: system has processor(s) with signature 0x000906ea

[Kyle Etsler (ketsler)]

I have replicated this issue on several systems. I have had the lockup occur at several moments at the lock screen. It usually locks up once you reach the login screen, sometimes when you start typing your password, sometimes when you hit enter after typing in a correct password.

I have replicated this issue on systems that have NVIDIA GPUs/Drivers, and those without.

The systems I have had this issue on have had 6th or 7th generation i7 Intel processors and all are laptops. I have a desktop with 8th generation Intel, with Linux Kernel 4.13.0-37 and the updated Intel microcode and I am not having this issue.

[Tyler Hicks (tyhicks)]

Could someone else with an affected system verify Erich's report that booting into the linux-image-4.13.0-36-generic kernel while using intel-microcode 3.20180312.0~ubuntu17.10.1 results in a system that does not experience lock ups?

[Eric Chandler (echandler)]

ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
CurrentDesktop: Unity
DistroRelease: Ubuntu 16.04
NonfreeKernelModules: nvidia_uvm nvidia_drm nvidia_modeset nvidia
Package: linux
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.13.0-37.42~16.04.1-generic 4.13.13
Tags: xenial
Uname: Linux 4.13.0-37-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: Domain Users adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True

[Eric Chandler (echandler)]

apport information

[Eric Chandler (echandler)]

apport information

[Eric Chandler (echandler)]

apport information

[Eric Chandler (echandler)]

apport information

[Eric Chandler (echandler)]

Booting with 4.13.0-36-generic prevented computer lockup during startup.

[John Bryant (bryantj247)]

I'm having the same problem. 4.13.0-36-generic works, 4.13.0-37-generic freezes up before I can log in. I have a 2nd generation core i7 and AMD graphics.

[Tyler Hicks (tyhicks)]

John, could you run 'apport-collect 1759920' or, alternatively, include your /proc/cpuinfo contents and the 'iucode-tool --scan-system' output?

[John Bryant (bryantj247)]

ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
CurrentDesktop: GNOME
DistroRelease: Ubuntu 17.10
InstallationDate: Installed on 2018-02-17 (40 days ago)
InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20180105.1)
Package: linux
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.13.0-36.40-generic 4.13.13
Tags: artful
Uname: Linux 4.13.0-36-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip libvirt lpadmin plugdev sambashare sudo
_MarkForUpload: True

[John Bryant (bryantj247)]

apport information

[John Bryant (bryantj247)]

apport information

[John Bryant (bryantj247)]

apport information

[John Bryant (bryantj247)]

apport information

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
CurrentDesktop: KDE
DistroRelease: KDE 16.04
InstallationDate: Installed on 2017-04-11 (352 days ago)
InstallationMedia: neon user "Xenial" - Build amd64 LIVE Binary 20170323-10:36
Package: linux
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 4.13.0-37.42~16.04.1-generic 4.13.13
Tags: third-party-packages xenial
Uname: Linux 4.13.0-37-generic x86_64
UnreportableReason: 這不是官方的 KDE 套件。請移除任何第三方套件並重試。
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm bumblebee cdrom dip docker libvirtd lpadmin lxd plugdev sambashare scanner sudo ubridge vboxusers video
_MarkForUpload: True

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

apport information

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

apport information

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

apport information

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

apport information

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

I have this issue as well, and I'm pretty sure this is a regression of the latest released intel-microcode package as I deliberately upgrade this single one just today.

[林博仁(Buo-ren, Lin) (buo-ren-lin)]

This is a Ivybridge based laptop, ASUS S550CB to be specific.

[Gordon Lack (gordon-lack)]

Bug #1760095 (mine) reports it for two 17.10 systems (Intel graphics only) both on 4.13.0-37-generic.
An 18.04beta system 4.15.0-13-generic with the "same" microcode version is OK.

[Erich E. Hoover (ehoover)]

I've identified a couple more systems that are affected by this issue:
iucode-tool: system has processor(s) with signature 0x00040651
iucode-tool: system has processor(s) with signature 0x000806e9

That second one was actually having trouble with 4.4.0-116-generic (boots fine with 4.4.0-112-generic). I installed the HWE package on the system and it will not work with either 4.13.0-37-generic or 4.13.0-36-generic, but it will work on 4.13.0-37-generic with intel-microcode 3.20180108.0+really20170707ubuntu16.04.1.

[Hannu Koivisto (azure)]

Kubuntu 17.10, kernel 4.13.0-37-generic, Dell E6330 with Intel graphics (iucode-tool: system has processor(s) with signature 0x000306a9, cpuinfo attached), after intel-microcode upgrade to 3.20180312.0~ubuntu17.10.1 => lockup during boot after entering disk encryption password but before login screen appears.

Boot with 4.13.0-36-generic => no problem.

[Tyler Hicks (tyhicks)]

I'm still unable to reproduce these lock ups and I've tested on three different systems. So, I'm reliant on you (very helpful) bug reporters.

Could someone test out the 4.13.0-38-generic kernel from artful-proposed to see if you experience any lock ups with the 3.20180312.0 microcode?

In the meantime, I'm looking through the changes between 4.13.0-36 and 4.13.0-37 to see if there's anything that sticks out. Nearly all of the changes are ARM related so that limits the possibilities in a nice way.

[Gordon Lack (gordon-lack)]

> Could someone test out the 4.13.0-38-generic kernel from artful-proposed
> to see if you experience any lock ups with the 3.20180312.0 microcode?

Any simple install instructions?

[Tyler Hicks (tyhicks)]

On 03/30/2018 10:59 AM, Gordon Lack wrote:
>> Could someone test out the 4.13.0-38-generic kernel from artful-proposed
>> to see if you experience any lock ups with the 3.20180312.0 microcode?
>
> Any simple install instructions?

Here are the typical install instructions for enabling proposed:

  https://wiki.ubuntu.com/Testing/EnableProposed

Be sure to follow the "Selective upgrading from -proposed" section.

[Gordon Lack (gordon-lack)]

For me, on my laptop (Intel(R) Core(TM) i5-4330M CPU @ 2.80GHz) with Kubuntu 17.10, the 4.13.0-38-generic kernel (signed image) has the same problem.
In fact it was slightly worse, a it seems to have wiped out my Ubuntu UEFI booting (boots straight to Windows - I can get to the Ubuntu menu from there and that works).
Now fixing that...

Reverting to the older intel-microcode with 4.13.0-38-generic is OK.

[QkiZ (qkiz)]

Works fine.
 ~  cat /proc/version_signature
Ubuntu 4.13.0-38.43+lp1759920.1-lowlatency 4.13.16
 ~  cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.13.0-38-lowlatency root=/dev/mapper/system-root ro rootflags=subvol=@ splash cryptopts=target=pandora-sys,source=/dev/disk/by-uuid/873980ab-cb18-419b-9614-54f23972e042,lvm=system elevator=deadline acpi_rev_override=1 vt.handoff=7
 ~  dmesg -t | grep -i microcode
microcode: microcode updated early to revision 0x24, date = 2018-01-21
microcode: sig=0x306c3, pf=0x20, revision=0x24
microcode: Microcode Update Driver: v2.2.

[Tyler Hicks (tyhicks)]

Any affected 16.04 users that are using the 4.4 kernel can find a test build with the fix here:

https://people.canonical.com/~tyhicks/lp1759920/xenial/

When you provide testing feedback, please remember to include the output from these commands in your comment:

$ cat /proc/version_signature
$ cat /proc/cmdline
$ dmesg -t | grep -i microcode

[John Bryant (bryantj247)]

I've been using the test kernel for 17.10 for a few hours, and it works great!

Ubuntu 4.13.0-38.43+lp1759920.1-generic 4.13.16

BOOT_IMAGE=/boot/vmlinuz-4.13.0-38-generic root=UUID=dd134dc1-6642-41d8-acd1-abc83ab24099 ro quiet splash vt.handoff=7

microcode: microcode updated early to revision 0x2d, date = 2018-02-07
microcode: sig=0x206a7, pf=0x2, revision=0x2d
microcode: Microcode Update Driver: v2.2.

[Kyle Etsler (ketsler)]

@tyhicks

I can confirm that adding 'apparmor=0' to the kernal boot parameters resolved the issue on an effected system.

This was tested on 4.13.0-37-generic, with intel microcode 3.20180312.0

I will proceed with testing kernel: need linux-image-4.13.0-38-generic_4.13.0-38.43+lp1759920.1

[Kyle Etsler (ketsler)]

@tyhicks

I can also confirm that the issue appears to be resolved with linux-image-4.13.0-38-generic_4.13.0-38.43+lp1759920.1

Below is the requested information.

$ uname -a
Linux mercury 4.13.0-38-generic #43+lp1759920.1 SMP Tue Apr 3 22:59:23 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ cat /proc/version_signature
Ubuntu 4.13.0-38.43+lp1759920.1-generic 4.13.16

$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.13.0-38-generic root=UUID=5dd1c110-663e-4cf0-9a63-45815590a186 ro quiet splash vt.handoff=7

$ dmesg -t | grep -i microcode
microcode: microcode updated early to revision 0x84, date = 2018-01-21
microcode: sig=0x906e9, pf=0x20, revision=0x84
microcode: Microcode Update Driver: v2.2.

[Aalok Sathe (aalok-sathe)]

This bug has been also reported elsewhere. See here, for instance: https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/1761121

Marked the other link as duplicate of this.

[Hannu Koivisto (azure)]

I can also confirm that the problem doesn't occur with linux-image-4.13.0-38-generic_4.13.0-38.43+lp1759920.1_amd64.deb

cat /proc/version_signature =>
Ubuntu 4.13.0-38.43+lp1759920.1-generic 4.13.16

cat /proc/cmdline =>
BOOT_IMAGE=/vmlinuz-4.13.0-38-generic root=/dev/mapper/kubuntu--vg-root ro quiet splash vt.handoff=7

dmesg -t | grep -i microcode =>
microcode: microcode updated early to revision 0x1f, date = 2018-02-07
microcode: sig=0x306a9, pf=0x10, revision=0x1f
microcode: Microcode Update Driver: v2.2.

[Maxim Loparev (laplandersan)]

Thanks @tyhicks.

$ dmesg | grep microcode
[ 0.000000] microcode: microcode updated early to revision 0x1f, date = 2018-02-07
[ 0.843339] microcode: sig=0x306a9, pf=0x2, revision=0x1f
[ 0.843412] microcode: Microcode Update Driver: v2.2.
$ cat /proc/version_signature
Ubuntu 4.13.0-38.43+lp1759920.1-generic 4.13.16
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.13.0-38-generic root=/dev/mapper/lmvg-ubunturoot ro zswap.enabled=0 quiet

[Rob Johnson (rob.johnson)]

I've tested Ty's 4.4 kernel from comment #78 and can confirm it has let me log in with an LDAP user.
Uname:
  Linux M3520 4.4.0-119-generic #143+lp1759920.1 SMP Wed Apr 4 21:32:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

version_signature:
  Ubuntu 4.4.0-119.143+lp1759920.1-generic 4.4.114

cmdline:
  BOOT_IMAGE=/vmlinuz-4.4.0-119-generic root=/dev/mapper/ubuntu--vg-root ro

dmesg:
microcode: CPU0 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU1 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU2 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU3 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU4 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU5 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU6 sig=0x906e9, pf=0x20, revision=0x84
microcode: CPU7 sig=0x906e9, pf=0x20, revision=0x84
microcode: Microcode Update Driver: v2.01 <email address hidden>, Peter Oruba

[QkiZ (qkiz)]

@Tyler when we should expect this fix in official Ubuntu repository?

[Damian Kidd (demomanca)]

Can confirm test kernal fixed my hard lock issue as well:
/proc/version_signature
Ubuntu 4.13.0-38.43+lp1759920.1-generic 4.13.16
/proc/cmdline
BOOT_IMAGE=/@/boot/vmlinuz-4.13.0-38-generic root=UUID=7dcb6f6f-1069-4315-a421-0d60fffecef9 ro rootflags=subvol=@
dmesg -t | grep -i microcode
microcode: microcode updated early to revision 0x2d, date = 2018-02-07
microcode: sig=0x206a7, pf=0x2, revision=0x2d
microcode: Microcode Update Driver: v2.2.

[Tyler Hicks (tyhicks)]

On 04/05/2018 03:25 PM, QkiZ wrote:
> @Tyler when we should expect this fix in official Ubuntu repository?
A little over 2 weeks from now. The week of April 23rd is the current
target.

[Jason P. Thomas (jthomasp)]

Is there any issue with removing the intel-microcode package with the -38 kernel? The intel-microcode package does not seem to get installed on a fresh 16.04.4 system and the Spectre/Meltdown checker referenced earlier shows no vulnerability as the mitigation is all done in the kernel.

[Paul Ashbrook (ashbrook)]

Tyler's amended kernel got me out of a hole today, thanks for the work. I've told my users not to update kernel until this is in the Ubuntu repos.

One observation from me is that VirtualBox 5.2.8 causes my HP Z230 (running Ubuntu 16.04.4) to crash and reboot when I try to start a VM. Not a showstopper for me, and I need to dig around for some clues, but wondered if anyone else had this problem.

[Michael Mauch (michael-mauch)]

Thank you, Tyler, your 4.4 kernel has been working great for half a day, even with the DVB-related patches for my DVB card.

viscacha% cat /proc/version_signature
Ubuntu 4.4.0-119.143+lp1759920.1-generic 4.4.114
viscacha% cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.4.0-119-generic root=UUID=009b027c-db85-4cd1-909a-a658aaab5394 ro nodmraid enable_mtrr_cleanup quiet splash ignore_loglevel i8042.noaux=1 video=vesafb:off intel_idle.max_cstate=1 console=ttyS0,115200 vt.handoff=7
viscacha% dmesg -t | grep -i microcode
microcode: CPU0 microcode updated early to revision 0x2d, date = 2018-02-07
microcode: CPU1 microcode updated early to revision 0x2d, date = 2018-02-07
microcode: CPU2 microcode updated early to revision 0x2d, date = 2018-02-07
microcode: CPU3 microcode updated early to revision 0x2d, date = 2018-02-07
microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU1 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU2 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU3 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU4 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU5 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU6 sig=0x206a7, pf=0x2, revision=0x2d
microcode: CPU7 sig=0x206a7, pf=0x2, revision=0x2d
microcode: Microcode Update Driver: v2.01 <email address hidden>, Peter Oruba

[Laurent Dinclaux (laurent-m)]

I uninstalled package intel-microcode and can now use my laptop.

[pinus (pinus)]

After purging intel-microcode package the 38 kernel runs for me.

[Kleber Sacilotto de Souza (kleber-souza)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'. If the problem still exists, change the tag 'verification-needed-trusty' to 'verification-failed-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Erich E. Hoover (ehoover)]

@kleber-souza I don't think that I have anything running trusty that I can test with, but I can easily test on xenial....

[Damian Kidd (demomanca)]

HWE Kernel in xenial-testing resolves the issue as well

/proc/version_signature
Ubuntu 4.13.0-39.44~16.04.1-generic 4.13.16
/proc/cmdline
BOOT_IMAGE=/@/boot/vmlinuz-4.13.0-39-generic root=UUID=7dcb6f6f-1069-4315-a421-0d60fffecef9 ro rootflags=subvol=@
dmesg -t | grep -i microcode
microcode: microcode updated early to revision 0x2d, date = 2018-02-07
microcode: sig=0x206a7, pf=0x2, revision=0x2d
microcode: Microcode Update Driver: v2.2.

[John Bryant (bryantj247)]

I wasn't able to reproduce the problem on Trusty, even with sssd installed.

/proc/version_signature
Ubuntu 4.4.0-119.143~14.04.1-generic 4.4.114

/proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.4.0-119-generic root=UUID=e57a614e-ad1a-449e-954d-ce832d128920 ro quiet splash vt.handoff=7

dmesg -t | grep -i microcode (with CPUs other than 0 removed)
microcode: CPU0 microcode updated early to revision 0x2d, date = 2018-02-07
microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x2d
microcode: Microcode Update Driver: v2.01 <email address hidden>, Peter Oruba

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-artful' to 'verification-done-artful'. If the problem still exists, change the tag 'verification-needed-artful' to 'verification-failed-artful'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Janne Snabb (janne-snabb)]

I tested the new kernel in artful-proposed on a system which failed with the newest kernel + intel-microcode in normal artful repository.

Thus I confirm that the kernel in artful-proposed fixes this issue for me.

$ apt policy linux-image-4.13.0-39-generic
linux-image-4.13.0-39-generic:
  Installed: 4.13.0-39.44
  Candidate: 4.13.0-39.44
  Version table:
 *** 4.13.0-39.44 400
        400 http://archive.ubuntu.com/ubuntu artful-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

$ apt policy intel-microcode
intel-microcode:
  Installed: 3.20180312.0~ubuntu17.10.1
  Candidate: 3.20180312.0~ubuntu17.10.1
  Version table:
 *** 3.20180312.0~ubuntu17.10.1 500
        500 http://fi.archive.ubuntu.com/ubuntu artful-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu artful-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.20170707.1 500
        500 http://fi.archive.ubuntu.com/ubuntu artful/restricted amd64 Packages

[pdf (pdffs)]

Tested kernel from xenial-proposed on an affected system, and the issue there appears to be resolved.

$ uname -r
4.4.0-120-generic

$ apt policy linux-image-generic
linux-image-generic:
  Installed: 4.4.0.120.126
  Candidate: 4.4.0.120.126
  Version table:
 *** 4.4.0.120.126 400
        400 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     4.4.0.119.125 500
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial-security/main amd64 Packages
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial-updates/main amd64 Packages
     4.4.0.21.22 500
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial/main amd64 Packages

$ apt policy intel-microcode
intel-microcode:
  Installed: 3.20180312.0~ubuntu16.04.1
  Candidate: 3.20180312.0~ubuntu16.04.1
  Version table:
 *** 3.20180312.0~ubuntu16.04.1 500
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial-security/main amd64 Packages
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     3.20151106.1 500
        500 http://mirror.internode.on.net/pub/ubuntu/ubuntu xenial/restricted amd64 Packages

[Gordon Lack (gordon-lack)]

I've tested the artful kernel on a UEFI system (and tracked own how the signed kernel is produced from the unsigned on in the debs at he same time - see #76).

It fixes the issue (tag already changed).

[Rob Johnson (rob.johnson)]

@brad-figg #98:
I've tested the Xenial-proposed kernel on a machine that was previously broken (even performed a fresh install to this effect). The proposed kernel (4.13.0-39) fixes the problem.

[Janne Snabb (snabb)]

I may have experienced this issue also on bionic (18.04) with the current kernel version as of today (4.15.0-13.14).

I upgraded one affected system with "do-release-upgrade -d". After completing the upgrade and rebooting, the system froze the same way as with the affected 17.10 kernel and microcode combination. After adding "apparmor=0" to the kernel command line the boot was successful. The system has sssd.

I was not able to perform further testing today. I will try to confirm tomorrow.

Or maybe someone else with an affected system is able to try bionic as well? ISOs available at http://releases.ubuntu.com/18.04/ if needed.

[Gordon Lack (gordon-lack)]

> I may have experienced this issue also on bionic (18.04) with the current kernel version as of today (4.15.0-13.14).

I have a system running bionic and it's fine (see #34). Always has been. It's never run anything else (just over 2 weeks old...). It's always had the "new" (3.20180312.0~ubuntu18.04.1) microcode.
It's running nightly auto-updates.

[Tyler Hicks (tyhicks)]

On 04/10/2018 11:43 AM, Janne Snabb wrote:
> I may have experienced this issue also on bionic (18.04) with the
> current kernel version as of today (4.15.0-13.14).

It wouldn't be this same exact bug as the 4.15.0-13.14 kernel doesn't
call __ptrace_may_access() when deciding to make use of IBPB while doing
a task switch.

It makes me wonder if you didn't accidentally reboot into the 17.10
kernel by accident (or maybe due to some post upgrade bug).

Either way, please leave a comment if it happens again. Thanks for
mentioning it here.

[Michael Mulqueen (michael.mulqueen)]

Is a lack of testing for trusty going to prevent a patch for xenial/artful? We don't have any machines on trusty to test it with, but if it's going to block the whole patch then we might be able to wipe an affected machine and install trusty to test it.

[Dimitri Papadopoulos (dimitri-papadopoulos)]

Experiencing freezes on multiple Ubuntu 16.04 LTS desktops of my organization, all using sssd. Freezes started around April 9th, but we usually deploy new packages after a time period of 5 to 7 days. I have experienced freezes with kernels 4.4.0-119, 4.4.40-116 and even 4.4.0-112 on my own desktop. No more freezes after upgrading to kernel 4.4.0-120.

[Tyler Hicks (tyhicks)]

I was never able to reproduce this issue on Ubuntu 14.04 but, after reviewing the Spectre V2 mitigations in the 3.13 based 14.04 kernel, I think the 14.04 kernel could be affected by this bug. Additionally, I didn't want the decision on when to make use of IBPB to be different in 14.04 than in all of the newer Ubuntu releases (16.04, 17.10, and soon to be 18.04). That's why I marked this bug as affecting 14.04 and submitted a fix for that kernel.

That being said, I don't think that we have any 14.04 users watching this bug so I did some SRU verification work myself. I installed sssd, the 3.13.0-145.194 kernel from trusty-proposed, and experienced no issues. I verified that the /proc/sys/kernel/ibpb_enabled file reported '1', indicating that IBPB was enabled. I also verified that the IBRS/IBPB/retpoline/spectre messages emitted from the kernel at boot time were as-expected. Retpoline was present, IBRS was disabled (due to retpoline), and IBPB was enabled.

[berend (berenddeboer)]

I can confirm that kernel in artful-proposed fixed the issue for me.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.13.0-145.194

---------------
linux (3.13.0-145.194) trusty; urgency=medium

  * linux: 3.13.0-145.194 -proposed tracker (LP: #1761430)

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel)
    - Revert "UBUNTU: SAUCE: x86/mm: Only set IBPB when the new thread cannot
      ptrace current thread"
    - x86/speculation: Use Indirect Branch Prediction Barrier in context switch

  * DKMS driver builds fail with: Cannot use CONFIG_STACK_VALIDATION=y, please
    install libelf-dev, libelf-devel or elfutils-libelf-devel (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

  * retpoline hints: primary infrastructure and initial hints (LP: #1758856)
    - [Packaging] retpoline-extract: flag *0xNNN(%reg) branches
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool on 32bit
    - x86/paravirt, objtool: Annotate indirect calls
    - x86/asm: Stop depending on ptrace.h in alternative.h
    - [Packaging] retpoline -- add safe usage hint support
    - [Packaging] retpoline-check -- only report additions
    - [Packaging] retpoline -- widen indirect call/jmp detection
    - [Packaging] retpoline -- elide %rip relative indirections
    - [Packaging] retpoline -- clear hint information from packages
    - SAUCE: modpost: add discard to non-allocatable whitelist
    - KVM: x86: Make indirect calls in emulator speculation safe
    - KVM: VMX: Make indirect call speculation safe
    - x86/boot, objtool: Annotate indirect jump in secondary_startup_64()
    - SAUCE: early/late -- annotate indirect calls in early/late initialisation
      code
    - SAUCE: vga_set_mode -- avoid jump tables
    - [Config] retpoline -- switch to new format
    - [Packaging] retpoline hints -- handle missing files when RETPOLINE not
      enabled
    - [Packaging] final-checks -- remove check for empty retpoline files

  * retpoline: ignore %cs:0xNNN constant indirections (LP: #1752655)
    - [Packaging] retpoline -- elide %cs:0xNNNN constants on i386

  * Boot crash with Trusty 3.13 (LP: #1757193)
    - Revert "UBUNTU: SAUCE: x86, extable: fix uaccess fixup detection"
    - x86/mm: Expand the exception table logic to allow new handling options

  * Segmentation fault in ldt_gdt_64 (LP: #1755817) // CVE-2017-5754
    - x86/kvm: Rename VMX's segment access rights defines
    - x86/signal/64: Fix SS if needed when delivering a 64-bit signal

 -- Kleber Sacilotto de Souza <email address hidden> Thu, 05 Apr 2018 16:26:39 +0200

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-121.145

---------------
linux (4.4.0-121.145) xenial; urgency=medium

  * linux: 4.4.0-121.145 -proposed tracker (LP: #1763687)

  * Ubuntu-4.4.0-120.144 fails to boot on arm64* hardware (LP: #1763644)
    - [Config] arm64: disable BPF_JIT_ALWAYS_ON

linux (4.4.0-120.144) xenial; urgency=medium

  * linux: 4.4.0-120.144 -proposed tracker (LP: #1761438)

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel)
    - Revert "x86/mm: Only set IBPB when the new thread cannot ptrace current
      thread"
    - x86/speculation: Use Indirect Branch Prediction Barrier in context switch

  * DKMS driver builds fail with: Cannot use CONFIG_STACK_VALIDATION=y, please
    install libelf-dev, libelf-devel or elfutils-libelf-devel (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

  * retpoline hints: primary infrastructure and initial hints (LP: #1758856)
    - [Packaging] retpoline-extract: flag *0xNNN(%reg) branches
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool on 32bit
    - x86/paravirt, objtool: Annotate indirect calls
    - x86/asm: Stop depending on ptrace.h in alternative.h
    - [Packaging] retpoline -- add safe usage hint support
    - [Packaging] retpoline-check -- only report additions
    - [Packaging] retpoline -- widen indirect call/jmp detection
    - [Packaging] retpoline -- elide %rip relative indirections
    - [Packaging] retpoline -- clear hint information from packages
    - SAUCE: modpost: add discard to non-allocatable whitelist
    - KVM: x86: Make indirect calls in emulator speculation safe
    - KVM: VMX: Make indirect call speculation safe
    - x86/boot, objtool: Annotate indirect jump in secondary_startup_64()
    - SAUCE: early/late -- annotate indirect calls in early/late initialisation
      code
    - SAUCE: vga_set_mode -- avoid jump tables
    - [Config] retpoline -- switch to new format
    - [Packaging] final-checks -- remove check for empty retpoline files

  * Xenial update to 4.4.117 stable release (LP: #1756860)
    - IB/mlx4: Fix incorrectly releasing steerable UD QPs when have only ETH ports
    - PM / devfreq: Propagate error from devfreq_add_device()
    - s390: fix handling of -1 in set{,fs}[gu]id16 syscalls
    - ARM: dts: STi: Add gpio polarity for "hdmi,hpd-gpio" property
    - arm: spear600: Add missing interrupt-parent of rtc
    - arm: spear13xx: Fix dmas cells
    - arm: spear13xx: Fix spics gpio controller's warning
    - ALSA: seq: Fix regression by incorrect ioctl_mutex usages
    - KVM/x86: Reduce retpoline performance impact in slot_handle_level_range(),
      by always inlining iterator helper methods
    - x86/cpu: Change type of x86_cache_size variable to unsigned int
    - drm/radeon: adjust tested variable
    - rtc-opal: Fix handling of firmware error codes, prevent busy loops
    - ext4: save error to disk in __ext4_grp_locked_error()
    - ext4: correct documentation for grpid mount option
    - mm: hide a #warning fo...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.13.0-39.44

---------------
linux (4.13.0-39.44) artful; urgency=medium

  * linux: 4.13.0-39.44 -proposed tracker (LP: #1761456)

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2
    Intel) // CVE-2017-5754
    - x86/mm: Reinitialize TLB state on hotplug and resume

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel)
    - Revert "x86/mm: Only set IBPB when the new thread cannot ptrace current
      thread"
    - x86/speculation: Use Indirect Branch Prediction Barrier in context switch

  * DKMS driver builds fail with: Cannot use CONFIG_STACK_VALIDATION=y, please
    install libelf-dev, libelf-devel or elfutils-libelf-devel (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

  * retpoline hints: primary infrastructure and initial hints (LP: #1758856)
    - [Packaging] retpoline-extract: flag *0xNNN(%reg) branches
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool on 32bit
    - x86/paravirt, objtool: Annotate indirect calls
    - [Packaging] retpoline -- add safe usage hint support
    - [Packaging] retpoline-check -- only report additions
    - [Packaging] retpoline -- widen indirect call/jmp detection
    - [Packaging] retpoline -- elide %rip relative indirections
    - [Packaging] retpoline -- clear hint information from packages
    - KVM: x86: Make indirect calls in emulator speculation safe
    - KVM: VMX: Make indirect call speculation safe
    - x86/boot, objtool: Annotate indirect jump in secondary_startup_64()
    - SAUCE: early/late -- annotate indirect calls in early/late initialisation
      code
    - SAUCE: vga_set_mode -- avoid jump tables
    - [Config] retpoline -- switch to new format
    - [Packaging] retpoline hints -- handle missing files when RETPOLINE not
      enabled
    - [Packaging] final-checks -- remove check for empty retpoline files

  * retpoline: ignore %cs:0xNNN constant indirections (LP: #1752655)
    - [Packaging] retpoline -- elide %cs:0xNNNN constants on i386

  * zfs system process hung on container stop/delete (LP: #1754584)
    - SAUCE: Fix non-prefaulted page deadlock (LP: #1754584)

  * zfs-linux 0.6.5.11-1ubuntu5 ADT test failure with linux 4.15.0-1.2
    (LP: #1737761)
    - SAUCE: (noup) Update zfs to 0.6.5.11-1ubuntu3.2

  * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10
    (LP: #1759312)
    - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features

  * btrfs and tar sparse truncate archives (LP: #1757565)
    - Btrfs: move definition of the function btrfs_find_new_delalloc_bytes
    - Btrfs: fix reported number of inode blocks after buffered append writes

  * efifb broken on ThunderX-based Gigabyte nodes (LP: #1758375)
    - drivers/fbdev/efifb: Allow BAR to be moved instead of claiming it

  * Intel i40e PF reset due to incorrect MDD detection (continues...)
    (LP: #1723127)
    - i40e/i40ev...

[bmaupin (bmaupin)]

This is *not* fixed for Xenial. I had originally applied the workaround in the bug description to get my computer to boot again. After seeing a "fixed" status for this bug, I updated intel-microcode. After I rebooted, my computer would not boot to the kernel which had the microcode applied (4.4.0-122). It would only boot to the previous kernel without the microcode (4.4.0-119).

$ tail -n 5 /var/log/apt/history.log
Start-Date: 2018-05-05 06:06:43
Commandline: apt install intel-microcode
Upgrade: intel-microcode:amd64 (3.20180108.0+really20170707ubuntu16.04.1, 3.20180312.0~ubuntu16.04.1)
End-Date: 2018-05-05 06:06:57

$ uname -a
Linux thinkpad 4.4.0-119-generic #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg -l | grep linux-image-4 | grep ^i
ii linux-image-4.4.0-112-generic 4.4.0-112.135 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-119-generic 4.4.0-119.143 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-122-generic 4.4.0-122.146 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 58
model name : Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz
stepping : 9
microcode : 0x1c
cpu MHz : 893.761
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb retpoline kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bugs : cpu_meltdown spectre_v1 spectre_v2
bogomips : 3391.94
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

[bmaupin (bmaupin)]

Since this bug was marked as fixed I'm not sure what the best way to proceed is. I went ahead and filed a new bug: https://bugs.launchpad.net/ubuntu/+source/intel-microcode/+bug/1769335

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.2.0-150.197

---------------
linux (3.2.0-150.197) precise; urgency=medium

  * precise/linux: 3.2.0-150.197 -proposed tracker (LP: #1919172)

  * CVE-2021-27365
    - scsi: iscsi: Verify lengths on passthrough PDUs
    - sysfs: Add sysfs_emit and sysfs_emit_at to format sysfs output
    - scsi: iscsi: Ensure sysfs attributes are limited to PAGE_SIZE

  * CVE-2021-27363 // CVE-2021-27364
    - scsi: iscsi: Restrict sessions and handles to admin capabilities

  * CVE-2021-27364
    - scsi: iscsi: respond to netlink with unicast when appropriate
    - Add file_ns_capable() helper function for open-time capability checking
    - net: Add variants of capable for use on on sockets
    - netlink: Make the sending netlink socket availabe in NETLINK_CB

 -- Thadeu Lima de Souza Cascardo <email address hidden> Mon, 05 Apr 2021 14:23:29 -0300