NULL pointer dereference at sd_revalidate_disk+0x30/0x2a0

[Seth Forshee]

I had just partitioned and formatted a USB memory stick using gparted and run sync. The bug happened when I pulled the memory stick out of the USB port.

BUG: unable to handle kernel NULL pointer dereference at
0000000000000008
IP: [<ffffffff8143fed0>] sd_revalidate_disk+0x30/0x2a0
PGD 168118067 PUD 168119067 PMD 0
Oops: 0000 [#1] SMP
CPU 3
Modules linked in: nls_iso8859_1 nls_cp437 vfat fat usb_storage uas dm_crypt parport_pc ppdev snd_hda_codec_hdmi snd_hda_codec_cirrus wl(P) rfcomm bnep lib80211 bcma arc4 snd_usb_audio snd_hda_intel snd_hda_codec snd_pcm uvcvideo videodev v4l2_compat_ioctl32 brcmsmac mac80211 brcmutil snd_hwdep snd_usbmidi_lib snd_seq_midi joydev btusb bluetooth snd_rawmidi snd_seq_midi_event bcm5974 snd_seq snd_timer snd_seq_device snd soundcore applesmc snd_page_alloc input_polldev cfg80211 crc8 cordic mei(C) apple_bl lp parport hid_apple i915 drm_kms_helper drm i2c_algo_bit usbhid hid video

Pid: 1653, comm: udisks-daemon Tainted: P C O 3.2.0-8-generic #14-Ubuntu Apple Inc. MacBookAir4,1/Mac-C08A6BB70A942AC2
RIP: 0010:[<ffffffff8143fed0>] [<ffffffff8143fed0>] sd_revalidate_disk+0x30/0x2a0
RSP: 0018:ffff880168385b08 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88016358b940
RDX: 0000000000000005 RSI: 0000000000000002 RDI: ffff880167fa8800
RBP: ffff880168385b28 R08: 0000000800000000 R09: 00000008ffffffff
R10: 0000000000000000 R11: ffffc90000406000 R12: ffff880167fa8800
R13: 00000000ffffff85 R14: ffff8801665a7b98 R15: ffff880167fa8800
FS: 00007f0d7ca597c0(0000) GS:ffff88016fac0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000008 CR3: 000000016436f000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process udisks-daemon (pid: 1653, threadinfo ffff880168384000, task ffff880168728000)
Stack:
 ffff8801665a7b80 ffff880167fa8800 00000000ffffff85 ffff8801665a7b98
 ffff880168385ba8 ffffffff811e294a ffff880167fa880c ffff880167fa8878
 0000000068385b68 ffff8801665a7b80 ffff880167fab400 ffff880167fa8800
Call Trace:
 [<ffffffff811e294a>] rescan_partitions+0xaa/0x300
 [<ffffffff811ae95c>] __blkdev_get+0x2bc/0x420
 [<ffffffff811891f0>] ? __pollwait+0xf0/0xf0
 [<ffffffff811aeb1e>] blkdev_get+0x5e/0x1e0
 [<ffffffff811aecfd>] blkdev_open+0x5d/0x80
 [<ffffffff81173f10>] __dentry_open+0x290/0x360
 [<ffffffff811aeca0>] ? blkdev_get+0x1e0/0x1e0
 [<ffffffff8129a2ec>] ? security_inode_permission+0x1c/0x30
 [<ffffffff8117560d>] nameidata_to_filp+0xad/0xb0
 [<ffffffff811844d8>] do_last+0x3f8/0x730
 [<ffffffff81185bb1>] path_openat+0xd1/0x3f0
 [<ffffffff81308a07>] ? kobject_put+0x27/0x60
 [<ffffffff813ec147>] ? put_device+0x17/0x20
 [<ffffffff81185ff2>] do_filp_open+0x42/0xa0
 [<ffffffff81314ba1>] ? strncpy_from_user+0x31/0x40
 [<ffffffff8118134a>] ? do_getname+0x10a/0x180
 [<ffffffff8165208e>] ? _raw_spin_lock+0xe/0x20
 [<ffffffff81193277>] ? alloc_fd+0xf7/0x150
 [<ffffffff811756fd>] do_sys_open+0xed/0x220
 [<ffffffff81175850>] sys_open+0x20/0x30
 [<ffffffff8165a5c2>] system_call_fastpath+0x16/0x1b
Code: 83 ec 20 48 89 5d e0 4c 89 65 e8 4c 89 6d f0 4c 89 75 f8 66 66 66 66 90 8b 05 1d 86 ad 00 48 8b 9f 38 03 00 00 49 89 fc c1 e8 15 <4c> 8b 6b 08 83 e0 07 83 f8 03 0f 87 0d 02 00 00 41 8b 85 70 06
RIP [<ffffffff8143fed0>] sd_revalidate_disk+0x30/0x2a0
 RSP <ffff880168385b08>
CR2: 0000000000000008

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: linux-image-3.2.0-8-generic 3.2.0-8.14
ProcVersionSignature: Ubuntu 3.2.0-8.14-generic 3.2.0
Uname: Linux 3.2.0-8-generic x86_64
NonfreeKernelModules: wl
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 1.90-0ubuntu1
Architecture: amd64
ArecordDevices:
 **** List of CAPTURE Hardware Devices ****
 card 0: PCH [HDA Intel PCH], device 0: Cirrus Analog [Cirrus Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: sforshee 3129 F.... pulseaudio
Card0.Amixer.info:
 Card hw:0 'PCH'/'HDA Intel PCH at 0xa0600000 irq 47'
   Mixer name : 'Intel CougarPoint HDMI'
   Components : 'HDA:10134206,106b6200,00100302 HDA:80862805,80860101,00100000'
   Controls : 25
   Simple ctrls : 9
Date: Tue Jan 10 16:02:48 2012
EcryptfsInUse: Yes
HibernationDevice: RESUME=UUID=df8148cd-dd0f-4ab0-8c7f-95b373e85e2c
InstallationMedia: Ubuntu 12.04 LTS "Precise Pangolin" - Alpha amd64+mac (20111208)
MachineType: Apple Inc. MacBookAir4,1
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.2.0-8-generic root=UUID=f4af0efe-df4b-4d74-8995-7081cf79889c ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.2.0-8-generic N/A
 linux-backports-modules-3.2.0-8-generic N/A
 linux-firmware 1.67
SourcePackage: linux
StagingDrivers: mei
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 10/14/2011
dmi.bios.vendor: Apple Inc.
dmi.bios.version: MBA41.88Z.0077.B0E.1110141154
dmi.board.asset.tag: Base Board Asset Tag#
dmi.board.name: Mac-C08A6BB70A942AC2
dmi.board.vendor: Apple Inc.
dmi.board.version: MacBookAir4,1
dmi.chassis.type: 10
dmi.chassis.vendor: Apple Inc.
dmi.chassis.version: Mac-C08A6BB70A942AC2
dmi.modalias: dmi:bvnAppleInc.:bvrMBA41.88Z.0077.B0E.1110141154:bd10/14/2011:svnAppleInc.:pnMacBookAir4,1:pvr1.0:rvnAppleInc.:rnMac-C08A6BB70A942AC2:rvrMacBookAir4,1:cvnAppleInc.:ct10:cvrMac-C08A6BB70A942AC2:
dmi.product.name: MacBookAir4,1
dmi.product.version: 1.0
dmi.sys.vendor: Apple Inc.

[Joseph Salisbury (jsalisbury)]

I got this same trace when disconnecting my phone from the USB port. My system is a Lenovo x201.

[Brad Figg (brad-figg)]

Thank you for taking the time to file a bug report on this issue.

However, given the number of bugs that the Kernel Team receives during any development cycle it is impossible for us to review them all. Therefore, we occasionally resort to using automated bots to request further testing. This is such a request.

We have noted that there is a newer version of the development kernel than the one you last tested when this issue was found. Please test again with the newer kernel and indicate in the bug if this issue still exists or not.

You can update to the latest development kernel by simply running the following commands in a terminal window:

    sudo apt-get update
    sudo apt-get upgrade

If the bug still exists, change the bug status from Incomplete to Confirmed. If the bug no longer exists, change the bug status from Incomplete to Fix Released.

If you want this bot to quit automatically requesting kernel tests, add a tag named: bot-stop-nagging.

 Thank you for your help, we really do appreciate it.

[Joseph Salisbury (jsalisbury)]

It appears this buug does not happen on my system when I'm using a docking station.

[Joseph Salisbury (jsalisbury)]

I would like to perform a bisect to identify the commit that caused this regression.

Does someone affected by this bug have a consistent way to reproduce the oops?

[Ming Lei (tom-leiming)]

I can reproduce the bug on the latest upstream kernel.

[Ming Lei (tom-leiming)]

The problem is discussing on upstream kernel mail list, and one patch
has been figured out
to address it.

http://marc.info/?t=132866137000002&r=1&w=2

[Ming Lei (tom-leiming)]

The latest acked patch for the issue:

http://www.spinics.net/lists/linux-scsi/msg57577.html

On Mon, Feb 20, 2012 at 10:05 PM, Ming Lei <email address hidden> wrote:
> The problem is discussing on upstream kernel mail list, and one patch
> has been figured out
> to address it.
>
> http://marc.info/?t=132866137000002&r=1&w=2

[Joseph Salisbury (jsalisbury)]

Thanks for the pointer, Ming. I'll build a test kernel with this patch and post a link shortly.

[Joseph Salisbury (jsalisbury)]

I created a set of test kernels, which are available at:
http://people.canonical.com/~jsalisbury/lp914319/

This test kernel has been patches with the patch mentioned in comment #8.

Can folks affected by this bug test this kernel and report back if it resolves this bug?

[Patrick Campanale (patrickappcreator)]

I have reverted back to 11.10 from 12.04 just because of some stability
issues with 12.04 at this current time. So I cannot test this kernel
anymore.
Sorry...

Thank you,
Patrick Campanale
Awesome Apps& Project Macaw

On 2/22/2012 2:00 PM, Joseph Salisbury wrote:
> I created a set of test kernels, which are available at:
> http://people.canonical.com/~jsalisbury/lp914319/
>
> This test kernel has been patches with the patch mentioned in comment
> #8.
>
> Can folks affected by this bug test this kernel and report back if it
> resolves this bug?
>

[Joseph Salisbury (jsalisbury)]

Is there anyone hitting this bug that can test the kernel mentioned in comment #10?

[Seth Forshee (sforshee)]

Unfortunately I've only seen it a couple of times, quite unpredictably, despite using USB storage fairly frequently. Even if I were running the kernel I'd be unable to assert that the problem was fixed with any confidence.

It looks like the patch isn't yet in Linus's tree, but there was some discussion about sending it on to stable that seemed positive, so it may be that we get the patch no matter what. As long as it hits Linus's tree soon though I'd think we ought to be able to get it into precise without too much trouble.

[Chris J Arges (arges)]

This bug hit me yesterday on 3.2.0-17.27 on my Lenovo T420. Will try the test kernel when I have time today.
Not sure how reliable the reproducer is, seems to happen occasionally when pulling out a USB stick.

[Ming Lei (tom-leiming)]

On Wed, Feb 29, 2012 at 10:07 PM, Chris J Arges
<email address hidden> wrote:
> This bug hit me yesterday on 3.2.0-17.27 on my Lenovo T420. Will try the test kernel when I have time today.
> Not sure how reliable the reproducer is, seems to happen occasionally when pulling out a USB stick.

I can reproduce it with continuous unplugging/plugging usb disk after starting
the below script. And looks the upstream patch can fix the issue
on my pandaboard.

#!/bin/sh
umount /media/*
while [ 1 ]
do
        fdisk -l /dev/sdb # suppose /dev/sdb is the usb disk
        sleep 0.05
done

thanks
--
Ming Lei

[Joseph Salisbury (jsalisbury)]

Thanks for the update, Ming. Did you test the upstream patch using the kernels mentioned in comment #10?

[Ming Lei (tom-leiming)]

Hi Joseph,

On Fri, Mar 2, 2012 at 12:36 AM, Joseph Salisbury
<email address hidden> wrote:
> Thanks for the update, Ming.  Did you test the upstream patch using the
> kernels mentioned in comment #10?

I tested the patch against -rc5-next on my pandaboard, and I am sure that
the patch does fix the problem on my board.

Looks the image you built is not installable, see below message:

[tom@~]$sudo dpkg -i
/home/tom/temp/test/linux-image-3.3.0-030300rc4-generic_3.3.0-030300rc4.201202221601_amd64.deb
Selecting previously unselected package linux-image-3.3.0-030300rc4-generic.
(Reading database ... 598800 files and directories currently installed.)
Unpacking linux-image-3.3.0-030300rc4-generic (from
.../linux-image-3.3.0-030300rc4-generic_3.3.0-030300rc4.201202221601_amd64.deb)
...
Examining /etc/kernel/preinst.d/
Done.
dpkg-deb (subprocess): short read on buffer copy for failed to write
to pipe in copy
dpkg-deb (subprocess): data: internal bzip2 read error: 'UNEXPECTED_EOF'
dpkg-deb: error: subprocess <decompress> returned error exit status 2
dpkg: error processing
/home/tom/temp/test/linux-image-3.3.0-030300rc4-generic_3.3.0-030300rc4.201202221601_amd64.deb
(--install):
 short read on buffer copy for backend dpkg-deb during
`./lib/modules/3.3.0-030300rc4-generic/kernel/drivers/video/backlight/adp5520_bl.ko'
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools
3.3.0-030300rc4-generic /boot/vmlinuz-3.3.0-030300rc4-generic
run-parts: executing /etc/kernel/postrm.d/zz-update-grub
3.3.0-030300rc4-generic /boot/vmlinuz-3.3.0-030300rc4-generic
Errors were encountered while processing:
 /home/tom/temp/test/linux-image-3.3.0-030300rc4-generic_3.3.0-030300rc4.201202221601_amd64.deb

thanks
--
Ming Lei

[Joseph Salisbury (jsalisbury)]

Thanks for the update, Ming. It looks like the .deb for my test kernel got corrupt:
dpkg-deb (subprocess): short read on buffer copy for failed to write
to pipe in copy

I'll build the kernel again, and re-post it.

[Joseph Salisbury (jsalisbury)]

@Ming

I created a new test kernel, which is available at:

http://people.canonical.com/~jsalisbury/lp914319/

[Ming Lei (tom-leiming)]

Hi Joseph,

On Tue, Mar 6, 2012 at 4:13 AM, Joseph Salisbury
<email address hidden> wrote:
> @Ming
>
> I created a new test kernel, which is available at:
>
> http://people.canonical.com/~jsalisbury/lp914319/

Looks the kernel images does fix the oops on my T410.

thanks,
--
Ming Lei

[Joseph Salisbury (jsalisbury)]

Thanks for testing, Ming.

I see the patches are queued up for Linus' tree, but haven't made it in yet:
https://lkml.org/lkml/2012/3/2/123

[Luis Henriques (henrix)]

This patch has hit mainline, with commit fe316bf2d5847bc5dd975668671a7b1067603bc7.

It may be a good idea to also take a look at commit 9f53d2fe815b4011ff930a7b6db98385d45faa68.

[Joseph Salisbury (jsalisbury)]

The patch is also queued up in stable-queue, with commit : 51e383c8660dec5ce16fb9da1937476c6b753885

In patch:
block-fix-null-pointer-dereference-in-sd_revalidate_disk.patch

[Joseph Salisbury (jsalisbury)]

This bug is fixed in the mainline 3.2.12 kernel.

The Ubuntu kernel has been rebased to 3.2.12 in version 3.2.0-19.31. Can folks affected by this bug upgrade to the latest Precise kernel and report back if this bug is resolved?

[Sascha (skbierm-deactivatedaccount)]

I already installed the new kernel, tried the same way it crashed before on my netbook - and it didn't, so looks solved for me.