[ASUS Sabertooth 990FX] Ubuntu 13.10/14.04 Catastrophic Random Kernel Crashes on Boot

[Michael Murphy]

The same issues are present both with GPT UEFI and simple MBR non-UEFI installs. I have installed Ubuntu 14.04 three times and each met the same fate. The first time you boot into the system after installing it, it works fine (maybe). However, after you reboot the system it will fail on such a grand catastrophic scale that you can't even boot into recovery mode since that will begin spamming error messages quickly as well. It does not output anything to logs during these errors so the only thing you have to go by are screenshots of the errors. In general the errors are all indicating issues with systemd-udevd. I also had similar problems in Ubuntu 13.10, plus some extra problems. Ubuntu 13.10 fails at installing grub and the grub-efi-amd64 and outputs tons of errors regarding efibootmgr.

It isn't related to the kernel since Ubuntu 13.04 with mainline kernel 3.12 works perfectly. It also doesn't seem to be related to firmware since I use linux-firmware-1.117 on Ubuntu 13.04 as well. It may be a bug related to efibootmgr and systemd.

---
ApportVersion: 2.9.2-0ubuntu8.5
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: mmstick 2398 F.... pulseaudio
 /dev/snd/controlC2: mmstick 2398 F.... pulseaudio
 /dev/snd/controlC0: mmstick 2398 F.... pulseaudio
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
DistroRelease: Ubuntu 13.04
InstallationDate: Installed on 2013-11-13 (40 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Release amd64 (20130424)
IwConfig:
 eth0 no wireless extensions.

 lo no wireless extensions.
MachineType: To be filled by O.E.M. To be filled by O.E.M.
MarkForUpload: True
NonfreeKernelModules: fglrx
Package: linux 3.8.0.34.52
PackageArchitecture: amd64
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 EFI VGA
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.8.0-34-generic root=UUID=1d566554-a73c-4ce0-9e48-45203b11c6f4 ro radeon.dpm=1 radeon.audio=1
ProcVersionSignature: Ubuntu 3.8.0-34.49-generic 3.8.13.12
RelatedPackageVersions:
 linux-restricted-modules-3.8.0-34-generic N/A
 linux-backports-modules-3.8.0-34-generic N/A
 linux-firmware 1.106
RfKill:

Tags: third-party-packages raring
Uname: Linux 3.8.0-34-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
dmi.bios.date: 10/16/2012
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1604
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: SABERTOOTH 990FX
dmi.board.vendor: ASUSTeK COMPUTER INC.
dmi.board.version: Rev 1.xx
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: To Be Filled By O.E.M.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1604:bd10/16/2012:svnTobefilledbyO.E.M.:pnTobefilledbyO.E.M.:pvrTobefilledbyO.E.M.:rvnASUSTeKCOMPUTERINC.:rnSABERTOOTH990FX:rvrRev1.xx:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:
dmi.product.name: To be filled by O.E.M.
dmi.product.version: To be filled by O.E.M.
dmi.sys.vendor: To be filled by O.E.M.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1250321/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Michael Murphy (mmstick)]

I'm getting the same in Ubuntu 13.10 with kernel 3.11 and 3.12.

bcrc32c firewire_ohci r8169 firewire_core mii crc_itu_t ahci libahci systemd-udevd tainted

[Quinn Balazs (qbalazs)]

Please attempt to attach the kern.log of the affected machine. You may be able to ssh in, and access the logs using "tail -f /var/log/kern.log".

[Michael Murphy (mmstick)]

Sorry, but you can't attach a log for a machine that won't even boot into recovery mode in the first place. I went back to Ubuntu 13.04 which is the only Ubuntu version that supports both my RadeonSI graphics card and my Sabertooth 990FX & AMD FX-8120 hardware. 13.10 and 14.04 just rapidly spit out errors with backtraces about firewire r8169 systemd-udevd tainted.

[Michael Murphy (mmstick)]

I'm guessing 12.04 through 13.04 doesn't crash because it always complains about r8169 having missing firmware when installing a kernel.

[Michael Murphy (mmstick)]

Contents of lspci

00:00.0 Host bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:02.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D)
00:05.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port E)
00:06.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port F)
00:09.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port H)
00:11.0 SATA controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices [AMD] nee ATI SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:15.0 PCI bridge: Advanced Micro Devices [AMD] nee ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
00:15.1 PCI bridge: Advanced Micro Devices [AMD] nee ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
00:16.0 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Tahiti PRO [Radeon HD 7950]
01:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI Tahiti XT HDMI Audio [Radeon HD 7970 Series]
02:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA Controller (rev 10)
03:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA Controller (rev 10)
04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
05:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
06:07.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (...

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the older firmware on Trusty(14.04)? This can be done by copying the firmware file from a 13.04 install to the 14.04 install. The file can be found at: /lib/firmware/rtlwifi/rtl8712u.bin

[Joseph Salisbury (jsalisbury)]

Or in your case, the rtl8712u.bin file may be under:
/lib/firmware/`uname -r`

[Michael Murphy (mmstick)]

I'm currently setting up a trible boot with Windows 8.1, Ubuntu 13.04, and an Ubuntu 14.04 Testing partition. I'll try that when I am done.

Are you sure I need rtl8712u.bin? I'm not aware of my Sabertooth 990FX having a Realtek wifi chip in it.

I do note that when I install a new kernel in Ubuntu 13.04 I get:

W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168g-3.fw for module r8169
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8106e-2.fw for module r8169
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8411-2.fw for module r8169

But I don't recall seeing that in Ubuntu 13.10 or 14.04.

[Michael Murphy (mmstick)]

Okay, the Live CD for Ubuntu 14.04 ran fine. However, after it rebooted it instantly hung as soon as it showed the Ubuntu desktop (with no icons on the panel) into my account (I had it set to autologin). Booting into recovery mode has it instantly start spitting text with a red background.

kern.log, boot.log, Xorg.0.log, syslog, everything is empty except dmesg.

Now I'll try moving firmwares around.

[Michael Murphy (mmstick)]

I tried copying all of rtl_nic from Raring to Trusty and this is what happened:

Boot #1: Worked Fine
Boot #2: No display, not even my mouse was turned on.
Boot #3: no display, but my mouse was turned on.
Boot to Recovery Mode #1: Hangs after adding hid-generic USB HID v1.11 Keyboard [Corsair Corsair M65 Gaming Mouse] and spams 'Bug Soft Lockup' for all of my cores.
Boot to Recovery Mode #2: Boots to the GUI screen but immediately crashes with a red background spitting out errors like 'systemd-udevd tainted' and 'kthreadd tainted'.

So rtl_nic firmwares may not be a part of the problem.

[Michael Murphy (mmstick)]

I tried booting with no devices connected to my system (not even a mouse or keyboard) and got the same results.

Boot #1: Boots then monitor says there is no display
Boot Recovery #1: Boots fine
Boot #2: Boots to desktop then immediately hangs (this time with icons in panel)
Boot Recovery #2 Instantly starts spitting out long stack traces

[Michael Murphy (mmstick)]

After replacing all of the firmware files in Trusty with Raring's, it still crashes. I'm not sure what else could be causing it.

[Michael Murphy (mmstick)]

Perhaps this is a bug with systemd-udev? after all, one of the errors I get is 'systemd-udevd tainted'

[Joseph Salisbury (jsalisbury)]

Can you follow the "Boot options" instructions on the following wiki to enable additional output on boot:

https://wiki.ubuntu.com/DebuggingKernelBoot

As mentioned on the wiki, it would be great if you can attach a log file which may have captured any messages you see. If you are unable to capture a log file, a digital photo will work just as well. As a last resort you can even copy messages down by hand.

[Michael Murphy (mmstick)]

Copying with a digital photo isn't that great since the error messages are so long that you can't see the entire error. You can't write it down either because the errors occur quickly.

[Michael Murphy (mmstick)]

There are some screenshots. I can't get any output from a regular boot since my monitor then says 'No Display' so I have to use recovery mode to show the error messages. I reinstalled Ubuntu 14.04 and set those boot options; and as usual the first time I boot into Ubuntu 13.10/14.04 it works fine. Then when I reboot the screen goes black and shows blue vertical lines with white horizontal rectangles and will never boot again.

[Michael Murphy (mmstick)]

Now that I think about it, I use the Oibaf PPA on Raring which includes linux-firmware 1.117 but Raring has always worked fine; so surely it can't be a firmware bug.

[Michael Murphy (mmstick)]

I notice that Raring does not have a file named systemd-udevd in /lib/systemd but Saucy/Trusting do. Therefore, the systemd-udevd thing is a new addition to Ubuntu starting with Saucy. It's the only thing I can think of that could be causing issues.

[Michael Murphy (mmstick)]

I'm currently on a live session of an Xubuntu 13.10 64-bit flash drive which I was attempting to install over my Ubuntu 14.04 partition and when it got to the installing grub2 portion it crashed and now I have a log I can provide.

[Michael Murphy (mmstick)]

Once again it has crashed at the same spot

Nov 15 11:13:27 xubuntu ubiquity: Setting up grub-efi-amd64-signed (1.22.1+2.00-19ubuntu2.1) ...
Nov 15 11:14:29 xubuntu kernel: [ 1035.084269] CPU: 7 PID: 15508 Comm: efibootmgr Tainted: GF D 3.11.0-12-generic #19-Ubuntu
Nov 15 11:14:29 xubuntu kernel: [ 1035.084465] [<ffffffff815b35fe>] efivar_entry_get+0x2e/0x70
Nov 15 11:14:29 xubuntu kernel: [ 1035.084474] [<ffffffff815b42bd>] efivar_show_raw+0x4d/0x130
Nov 15 11:14:29 xubuntu kernel: [ 1035.084481] [<ffffffff815b4862>] efivar_attr_show+0x42/0x60
*ensue crashing frenzy*

I'm not sure why the Xubuntu 13.10 ISO induces this crash when installing grub when Ubuntu 13.10/14.04 did not.

[Michael Murphy (mmstick)]

I don't see any mentions of systemd-udevd or kthreadd in these logs like I took in my screenshots (of which I don't remember seeing issues with efibootmgr) so the bug with Xubuntu may or may not be related to the issue I'm having here, although I would assume there is some sort of connection somehow. I should probably try installing Ubuntu 13.10 and seeing if it does the same thing Xubuntu 13.10 is doing.

[Joseph Salisbury (jsalisbury)]

If possible, can you test the latest mainline kernel to see if this bug is already fixed upstream? It can be downloaded from:

http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-trusty/

[Michael Murphy (mmstick)]

Ubuntu 14.04 already has the latest mainline kernel, and Ubuntu 13.04 works perfectly fine with every kernel from 3.8 to 3.12.

[Michael Murphy (mmstick)]

The issue with Xubuntu 13.10 seems to be related to this bug: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/879858

[Michael Murphy (mmstick)]

Currently in the Ubuntu 14.04 daily live ISO. As usual, it installs fine with no errors.

[Michael Murphy (mmstick)]

and upon trying to boot into Ubuntu 14.04, the systemd-udevd errors still prevent me to boot as usual.

[Michael Murphy (mmstick)]

and as always, not a single kern.log, syslog, or anything relevant is capture so the only thing you have to see, in regards to the stack traces, are the screenshots I provided earlier.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michael Murphy (mmstick)]

OpenSUSE 13.1 installs and runs fine so there is an issue with how Debian and/or Ubuntu is doing things here.

[Michael Murphy (mmstick)]

It seems Ubuntu 12.04.3 LTS also suffers the same fate that 13.10 does when installing grub to a GPT UEFI partition; an instant kernel panic.

[Nazar Mokrynskyi (nazar-pc)]

The same bug with such configuration:
AMD FX-8120
AsRock 990FX Extreme3

My custom kernels from 3.11 to latest 3.13 RC all works fine, but kernels 3.11+ on raring and trusty kernel crashes on start.

[Nazar Mokrynskyi (nazar-pc)]

Last crash occurred with latest beta fglrx, after removing system booted fine.

[Michael Murphy (mmstick)]

For me it wasn't related to the kernel at all. I'm currently running a Trusty install that I installed on another machine on my Sabertooth 990FX and it's been working fine. I haven't tried installing 13.10 or 14.04 from scratch yet though.

[penalvch (penalvch)]

Nazar Mokrynskyi, thank you for your comment. Given how your issue is resolved removing fglrx, this wouldn't be considered the same bug. Hence, so your hardware may be tracked, could you please file a new report by executing the following in a terminal while booted into a Ubuntu repository kernel (not a mainline one) via:
ubuntu-bug linux

For more on this, please read the official Ubuntu documentation:
Ubuntu Bug Control and Ubuntu Bug Squad: https://wiki.ubuntu.com/Bugs/BestPractices#X.2BAC8-Reporting.Focus_on_One_Issue
Ubuntu Kernel Team: https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Filing_Kernel_Bug_reports
Ubuntu Community: https://help.ubuntu.com/community/ReportingBugs#Bug_reporting_etiquette

When opening up the new report, please feel free to subscribe me to it.

Thank you for your understanding.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

My Raring install is using kernel 3.12. I'm planning on reformatting my SSD later and installing a fresh copy of Ubuntu 13.10 to see if that will work, but currently am just using a hard drive from a different PC that I installed Ubuntu 14.04 to.

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[Michael Murphy (mmstick)]

apport information

[penalvch (penalvch)]

Michael Murphy, thank you for performing the requested action. Just to clarify your Bug Description:
>"It isn't related to the kernel since Ubuntu 13.04 with kernel 3.12 works perfectly."

Could you please provide the direct URL source of the 3.12 kernel you used which worked perfectly?

[Michael Murphy (mmstick)]

All kernels, rc and stable, work from 3.8 -> 3.13. I wrote a script that automatically downloads the latest kernels from the mainline PPA ( http://pastebin.com/kR9c6ymH ) so I would always have the latest RCs for testing; at least, all kernels worked in 13.04.

Ubuntu 13.10 installed without UEFI would install fine, but would always fail to boot with 'systemd-udevd tainted' errors.
Ubuntu 13.10 installed with UEFI would fail to install when installing grub-efi.
Ubuntu 12.04.3 also suffers from the same grub-efi bug.
Ubuntu 13.04 installs fine with and without UEFI.
I'm assuming the current Ubuntu 14.04 works fine with my Sabertooth 990FX since I'm using it right now.

Other Linux distros worked perfectly fine with and without UEFI as well, like Sabayon 13.10, OpenSUSE 13.1, and Fedora 19. So the bug only seemed to affect Ubuntu.

[penalvch (penalvch)]

Michael Murphy, thank you for the requested information. It would appear patches/a patch applied to the downstream kernel is correlating to this issue. Moving back to linux (Ubuntu) as this was a mis-toggle on my end.

[Alberto Salvia Novella (es20490446e)]

Since this bug crashes the entire operating system, it has a priority of "critical".

[Michael Murphy (mmstick)]

As far as I know, it is only a bug with Ubuntu 13.10 now. Ubuntu 14.04 development branch works perfectly fine.

[penalvch (penalvch)]

Michael Murphy, would you need a backport to a release prior to 14.04, or may this be closed as Status Invalid?

[Michael Murphy (mmstick)]

I wouldn't know what you would need to backport in order for Ubuntu 13.10 to work considering how I don't even know why 13.10 didn't work in the first place. Whatever package(s) were broken in 13.10 are not broken in Ubuntu 14.04. I suppose as a start, efibootmgr is broken in Ubuntu 12.04 and Ubuntu 13.10.

[penalvch (penalvch)]

Michael Murphy, well knowing what specifically it would take is a nice to have, but not a need at this point. This boils down to would you personally/your infrastructure need a backport, in which case an investigation into what it would take to backport would occur, or did you intend to use 14.04 going forward?

[Michael Murphy (mmstick)]

Considering the fact that I'm always using bleeding edge hardware, I'm always looking to stay on bleeding edge versions of Linux. Therefore, all of my systems are running Ubuntu 14.04 and will continue to do so for the immediate future.

[penalvch (penalvch)]

Michael Murphy, this bug report is being closed due to your last comment https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1250321/comments/71 regarding this being fixed with an update. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.