kernel lockup when plugging in USB stick

[KIAaze]

I have a USB stick which sometimes causes the kernel to completely lock up when I plug it in on my desktop PC.
I noticed (but haven't strictly verified it yet) that it seems to happen whenever I used the USB stick on a windows machine before.
When it happens, all I can do is a hard reboot.
Nothing works, not even the magic sysrq keys. Caps lock has no effect, so I'm assuming a kernel lockup.

Kernel: 2.6.31-18-generic #55-Ubuntu SMP Fri Jan 8 14:54:52 UTC 2010 x86_64 GNU/Linux
USB stick: 8GB Transcend Jetflash 8GB
OS: Kubuntu 9.10

The same USB stick works without any problems on my laptop under windows+linux, as well as on my desktop PC with a live-CD (of Kubuntu 9.10).

Note:
I found this bug, which seems similar to the one I have, but with another kernel version: https://bugs.launchpad.net/bugs/322875

ProblemType: Bug
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: mike 2728 F.... knotify4
                      mike 2752 F.... pulseaudio
                      mike 2778 F.... amarok
                      mike 2804 F.... kmix
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info:
 Card hw:0 'SB'/'HDA ATI SB at 0xfe024000 irq 16'
   Mixer name : 'Realtek ALC888'
   Components : 'HDA:10ec0888,1458a002,00100001'
   Controls : 40
   Simple ctrls : 22
Date: Fri Jan 29 20:45:53 2010
DistroRelease: Ubuntu 9.10
HibernationDevice: RESUME=UUID=02b03bbd-b7e2-4ca4-8d01-aed5f9f0f650
MachineType: Gigabyte Technology Co., Ltd. GA-MA74GM-S2H
NonfreeKernelModules: nvidia
Package: linux-image-2.6.31-18-generic 2.6.31-18.55
ProcCmdLine: root=UUID=c33b4bf0-bf60-455e-bb7c-2c3c499582d3 ro
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-18.55-generic
RelatedPackageVersions:
 linux-backports-modules-2.6.31-18-generic N/A
 linux-firmware 1.26
RfKill:
 0: phy0: Wireless LAN
  Soft blocked: no
  Hard blocked: no
SourcePackage: linux
Uname: Linux 2.6.31-18-generic x86_64
WpaSupplicantLog:

XsessionErrors:
 (polkit-gnome-authentication-agent-1:2790): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (thunderbird-bin:2774): GLib-WARNING **: g_set_prgname() called multiple times
 (firefox-bin:3033): Gtk-WARNING **: /usr/lib/gtk-2.0/2.10.0/engines/libqtcurve.so: wrong ELF class: ELFCLASS64
dmi.bios.date: 06/02/2009
dmi.bios.vendor: Award Software International, Inc.
dmi.bios.version: F3
dmi.board.name: GA-MA74GM-S2H
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.modalias: dmi:bvnAwardSoftwareInternational,Inc.:bvrF3:bd06/02/2009:svnGigabyteTechnologyCo.,Ltd.:pnGA-MA74GM-S2H:pvr:rvnGigabyteTechnologyCo.,Ltd.:rnGA-MA74GM-S2H:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvr:
dmi.product.name: GA-MA74GM-S2H
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

[KIAaze (zohn-joidberg)]

Here is /var/log/messages after plugging in the USB stick while using the Kubuntu 9.10 live-CD.
The kernel does not lock up and everything works as expected.

[KIAaze (zohn-joidberg)]

Reinstalled Kubuntu 9.10. Problem hasn't reappeared yet, even after using USB stick on a windows machine before.

Just in case it's useful, here's a diff of fstab+mtab before and after reinstallation:
=======================
[~/kernel_hangs/diff]$ diff fstab_before fstab_after
3,5c3,5
< # Use 'vol_id --uuid' to print the universally unique identifier for a
< # device; this may be used with UUID= as a more robust way to name devices
< # that works even if disks are added and removed. See fstab(5).
---
> # Use 'blkid -o value -s UUID' to print the universally unique identifier
> # for a device; this may be used with UUID= as a more robust way to name
> # devices that works even if disks are added and removed. See fstab(5).
10c10
< UUID=c33b4bf0-bf60-455e-bb7c-2c3c499582d3 / ext4 relatime,errors=remount-ro 0 1
---
> UUID=0a03bba2-be0c-4f77-a393-dd39ff775ea9 / ext4 errors=remount-ro 0 1
12c12
< UUID=75eb4ad5-320f-4c6a-ae11-d0237997d55a /home ext4 relatime 0 2
---
> UUID=75eb4ad5-320f-4c6a-ae11-d0237997d55a /home ext4 defaults 0 2
[~/kernel_hangs/diff]$ diff mtab_before mtab_after_nousb
1c1
< /dev/sda2 / ext4 rw,relatime,errors=remount-ro 0 0
---
> /dev/sda2 / ext4 rw,errors=remount-ro 0 0
13c13,14
< /dev/sda5 /home ext4 rw,relatime 0 0
---
> none /var/lib/ureadahead/debugfs debugfs rw 0 0
> /dev/sda5 /home ext4 rw 0 0
=======================

mtab after plugging in the USB stick:
=======================
[~/kernel_hangs/diff]$ diff mtab_after_nousb mtab_after_usb
15a16
> /dev/sdb1 /media/disk vfat rw,nosuid,nodev,uhelper=hal,uid=1000,utf8,shortname=mixed,flush 0 0
=======================

[KIAaze (zohn-joidberg)]

Happened again, but with a different USB stick this time: a 4GB one.
And it never happened with smaller USB sticks so far.
Could this have to do with the 4GB filesize limit of FAT32 partitions?

[KIAaze (zohn-joidberg)]

Ok, removed some packages, which I suspect might cause the problem:
sudo apt-get remove --purge gisomount gmountiso gnome-mount

Just plugged in USB stick and it's still working. Will report back if the problem (which was kind of random) reappears.

[KIAaze (zohn-joidberg)]

It still hangs. :(
Even booting with the USB stick plugged in causes a kernel lock up.

Does anybody have any idea how I can debug this?

[KIAaze (zohn-joidberg)]

Still random lockups when plugging in USB stick.
There are 4 USB slots on the motherboard and 4 in the front, with mic and aux connectors.
I haven't checked how it's connected inside the PC yet.

Lockup also happened with a 1GB "Trekstor USB Stick SE".
Seems to happen most often when I use the front USB slots, but also happened at least once with one in the back (i.e. one directly on the motherboard).

One new thing I noticed, is that it now reboots after a while, when locked up.
After the reboot, the USB stick is usable.

[KIAaze (zohn-joidberg)]

Still happening after kernel updates by the way:
Linux my-desktop 2.6.31-20-generic #58-Ubuntu SMP Fri Mar 12 04:38:19 UTC 2010 x86_64 GNU/Linux

Partitions are all ext4, except windows and swap. I don't know if this matters.

Removed all nautilus stuff completely, except libnautilus-extension1 and libnautilus-extension-dev
Lock up still happening.

Will try these instructions once I have enough space and time: http://www.av8n.com/computer/htm/kernel-lockup.htm
Any suggestions are welcome...

I'm attaching the list of installed packages, since I haven't done it until now.

[KIAaze (zohn-joidberg)]

Things have changed: The magic sysrq keys now work when the system freezes!
I can now reboot with REISUB.
Any ideas how I can take advantage of the sysrq keys to unfreeze the system or at least get more debugging info?

I removed almost all *gnome* packages. Don't know if it has anything to do with it.

Maybe I should do more tests with Ubuntu Gnome or Windows to see if it happens there as well (hasn't been the case so far).

I also opened the PC to have a look at the connections. They seem to be OK.
The motherboard is a GIGABYTE GA-MA74GM-S2H.
The front USB ports are connected over some 9-pin connectors.
I'm adding some screenshots of the system.

I noticed that there were only 2 connectors on the motherboard for the 4 front USB ports (which all work when the system does not freeze) and they were on the "F_USB ports" 3 and 4, while 1 and 2 were empty.
Does this matter?

[KIAaze (zohn-joidberg)]

Ok...
Installed Ubuntu 9.10 (i.e. with Gnome) on a 5GB partition I had.
No freeze so far.

Also did a few tests on Windows XP and no freezes.

Is it possible that this problem is caused by KDE?

Another thing I noticed was this bug message in dmesg:
=====
[ 11.096177] EDAC amd64: This node reports that Memory ECC is currently disabled.
[ 11.096181] EDAC amd64: bit 0x400000 in register F3x44 of the MISC_CONTROL device (0000:00:18.3) should be enabled
[ 11.096183] EDAC amd64: WARNING: ECC is NOT currently enabled by the BIOS. Module will NOT be loaded.
[ 11.096185] Either Enable ECC in the BIOS, or use the 'ecc_enable_override' parameter.
[ 11.096186] Might be a BIOS bug, if BIOS says ECC is enabled
[ 11.096187] Use of the override can cause unknown side effects.
[ 11.096212] amd64_edac: probe of 0000:00:18.2 failed with error -22
=====
Could it have anything to do with my problem?
I even got this message while running the Ubuntu 9.10 Live-CD!

I found no ECC (error-correcting code?) setting in the BIOS.

Related bug reports:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/471814
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/422536

Note: It turns out the Ubuntu kernel comes with soft-lockup detection enabled, so recompiling is unnecessary.
At least "make menuconfig" showed it as enabled in the source gotten through apt-get.
Still, if recompiling with certain changes or settings is necessary, I'm willing to do it.
I just have absolutely no idea what to change or test right now.

[KIAaze (zohn-joidberg)]

Oh, and this is in dmesg too:
=====
[ 11.043488] nvidia: module license 'NVIDIA' taints kernel.
[ 11.043492] Disabling lock debugging due to kernel taint
=====
???

[KIAaze (zohn-joidberg)]

Switched to gdm/Gnome as default desktop since a few days (KDE software still there).
Haven't been able to reproduce the lock up + autoreboot despite several attempts.
So this seems to somehow be related to kdm/KDE.

However, I noticed strange behaviour in Gnome too:
-USB sticks don't always get automounted (with the nautilus browser window being opened automatically and the USB icon appearing on the desktop)
-They don't get unmounted correctly when using "safely remove" (I have /media/usb* directories left in /media)

[KIAaze (zohn-joidberg)]

Lockup+reboot happened again today, exactly when I needed to really use a USB key...
I noticed the following messages after the reboot (in tty1):
============
udevd[510]: can not read '/etc/udev/rules.d/z80_user.rules' (also present after "normal boot")
...
mountall: fsck /home[925] terminated with status 1
init: ureadahead-other main process (983) terminated with status 4 (also present after "normal boot")
============

z80_user.rules info:
============
[^_^][3][~]$ readlink -f /etc/udev/rules.d/z80_user.rules
/etc/udev/user.rules
[^_^][4][~]$ ls -l /etc/udev/rules.d/z80_user.rules
lrwxrwxrwx 1 root root 13 2010-02-03 15:14 /etc/udev/rules.d/z80_user.rules -> ../user.rules
[^_^][5][~]$ ls -l /etc/udev/user.rules
-rw-r--r-- 1 root root 0 2010-02-03 16:18 /etc/udev/user.rules
[^_^][6][~]$ file /etc/udev/user.rules
/etc/udev/user.rules: empty
[^_^][7][~]$ cat /etc/udev/user.rules
[^_^][8][~]$
============

[KIAaze (zohn-joidberg)]

Gnome also suddenly logged me out today while I was using Kate. And I have absolutely no idea why it happened. Some weird Gnome shortcut to log out???

I did install usbmount to try to fix the inconsistent usb key mounting behaviour. Could it have something to do with it? I have no idea...
If only I had time to start kernel hacking. :/

[KIAaze (zohn-joidberg)]

cf also http://ubuntuforums.org/showthread.php?t=1454806

[KIAaze (zohn-joidberg)]

Log files from last reboot into root mode after a crash caused by USB stick. (above one too)

[KIAaze (zohn-joidberg)]

Ok, I think I'll soon recompile my kernel with verbose power management debugging and whatever other debugging options there are... :/

[Jeremy Foshee (jeremyfoshee)]

Hi KIAaze,

If you could also please test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[KIAaze (zohn-joidberg)]

Ah! Finally an answer! :D
Will do so as soon as possible. I already have enough disk space to
compile now. :)
Any specific things I should configure in menuconfig?

On Tue, Apr 27, 2010 at 4:26 AM, Jeremy Foshee
<email address hidden> wrote:
> Hi KIAaze,
>
> If you could also please test the latest upstream kernel available that would be great.  It will allow additional upstream developers to examine the issue.  Refer to https://wiki.ubuntu.com/KernelMainlineBuilds .  Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag.  This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text.  Please let us know your results.
>
> Thanks in advance.
>
>    [This is an automated message.  Apologies if it has reached you inappropriately; please just reply to this message indicating so.]
>
>
> ** Tags added: needs-upstream-testing
>
> ** Tags added: kj-triage
>
> ** Changed in: linux (Ubuntu)
>       Status: New => Incomplete
>
> --
> kernel lockup when plugging in USB stick
> https://bugs.launchpad.net/bugs/514485
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Unlock your computing: http://www.getgnulinux.org/

[KIAaze (zohn-joidberg)]

Nevermind, no need to compile my own kernel apparently. :)

[Jeremy Foshee (jeremyfoshee)]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu development release http://cdimage.ubuntu.com/daily-live/current/ . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]