btrfs map_private_extent_buffer+0x12/0x150 NULL pointer dereference

[Colin Ian King]

While running a soak test I hit the following WARNING followed by a null pointer de-reference on btrfs inside a virtual machine.

$ uname -a
Linux server-7362 3.2.0-17-virtual #27-Ubuntu SMP Fri Feb 24 15:57:57 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

To repeat:

Start up a virtual machine

Create image:
dd if=/dev/zero of=image bs=1K count=280000
mkfs.btrfs image
gcc test.c -o test
sudo mount -o loop image /mnt
cd /mnt
~/test -d 120 foo

I can only get this to reproduce inside a virtual machine (such as an instance on the canonicloud). Seems like a race condition to me.

[ 4640.369358] Btrfs loaded
[ 4640.369487] device fsid 32c7ae64-51ff-4826-a581-c1930ce5c416 devid 1 transid 7 /dev/loop0
[ 4673.541599] ------------[ cut here ]------------
[ 4673.541620] WARNING: at /build/buildd/linux-3.2.0/fs/btrfs/extent-tree.c:4771 __btrfs_free_extent+0x5b0/0x650 [btrfs]()
[ 4673.541624] Hardware name: Bochs
[ 4673.541625] Modules linked in: btrfs zlib_deflate libcrc32c psmouse serio_raw virtio_balloon acpiphp floppy
[ 4673.541638] Pid: 21872, comm: test Not tainted 3.2.0-17-virtual #27-Ubuntu
[ 4673.541640] Call Trace:
[ 4673.541665] [<ffffffff81065dcf>] warn_slowpath_common+0x7f/0xc0
[ 4673.541670] [<ffffffff81065e2a>] warn_slowpath_null+0x1a/0x20
[ 4673.541681] [<ffffffffa0067f20>] __btrfs_free_extent+0x5b0/0x650 [btrfs]
[ 4673.541702] [<ffffffffa009b0a8>] ? extent_write_cache_pages.isra.21.constprop.31+0x108/0x3e0 [btrfs]
[ 4673.541714] [<ffffffffa00680d4>] run_delayed_tree_ref+0x114/0x1a0 [btrfs]
[ 4673.541726] [<ffffffff8115e6df>] ? kmem_cache_free+0x2f/0x110
[ 4673.541738] [<ffffffffa006bd3e>] run_one_delayed_ref+0xae/0xf0 [btrfs]
[ 4673.541750] [<ffffffffa006be54>] run_clustered_refs+0xd4/0x240 [btrfs]
[ 4673.541762] [<ffffffffa006c08a>] btrfs_run_delayed_refs+0xca/0x220 [btrfs]
[ 4673.541779] [<ffffffffa0096246>] ? btrfs_run_ordered_operations+0x1d6/0x1f0 [btrfs]
[ 4673.541794] [<ffffffffa007c113>] btrfs_commit_transaction+0x93/0x840 [btrfs]
[ 4673.541802] [<ffffffff810892b0>] ? add_wait_queue+0x60/0x60
[ 4673.541819] [<ffffffffa008ab77>] btrfs_sync_file+0x187/0x1f0 [btrfs]
[ 4673.541834] [<ffffffff811a2666>] do_fsync+0x56/0x80
[ 4673.541839] [<ffffffff811a29b3>] sys_fdatasync+0x13/0x20
[ 4673.541844] [<ffffffff8165a042>] system_call_fastpath+0x16/0x1b
[ 4673.541847] ---[ end trace dfc590b622064b16 ]---
[ 4673.541850] btrfs unable to find ref byte nr 29360128 parent 0 root 5 owner 0 offset 0
[ 4673.543048] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 4673.544081] IP: [<ffffffffa009d022>] map_private_extent_buffer+0x12/0x150 [btrfs]
[ 4673.544081] PGD 1c6ef067 PUD 1c649067 PMD 0
[ 4673.544081] Oops: 0000 [#1] SMP
[ 4673.544081] CPU 0
[ 4673.544081] Modules linked in: btrfs zlib_deflate libcrc32c psmouse serio_raw virtio_balloon acpiphp floppy
[ 4673.544081]
[ 4673.544081] Pid: 21872, comm: test Tainted: G W 3.2.0-17-virtual #27-Ubuntu Bochs Bochs
[ 4673.544081] RIP: 0010:[<ffffffffa009d022>] [<ffffffffa009d022>] map_private_extent_buffer+0x12/0x150 [btrfs]
[ 4673.544081] RSP: 0018:ffff88001ce2fb28 EFLAGS: 00010296
[ 4673.544081] RAX: 0000000000000000 RBX: 0000000000000065 RCX: ffff88001ce2fb58
[ 4673.544081] RDX: 0000000000000004 RSI: 000000000000007a RDI: 0000000000000000
[ 4673.544081] RBP: ffff88001ce2fb48 R08: ffff88001ce2fb60 R09: ffff88001ce2fb68
[ 4673.544081] R10: 0000000000000000 R11: 0000000000000000 R12: 000000000000007a
[ 4673.544081] R13: 0000000000000000 R14: 0000000000001000 R15: 00000000ffffffe4
[ 4673.544081] FS: 00007fbfcc84e700(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000
[ 4673.544081] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 4673.544081] CR2: 0000000000000000 CR3: 000000001caf7000 CR4: 00000000000006f0
[ 4673.544081] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4673.544081] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 4673.544081] Process test (pid: 21872, threadinfo ffff88001ce2e000, task ffff88001c9316c0)
[ 4673.544081] Stack:
[ 4673.544081] 0000000000001000 0000000000000065 000000000000007a 0000000000000000
[ 4673.544081] ffff88001ce2fb98 ffffffffa00927cc ffff88001ce2fba8 ffff88001ce2fb68
[ 4673.544081] 0000000000000005 0000000001c00000 0000000000000000 ffff88000475a090
[ 4673.544081] Call Trace:
[ 4673.544081] [<ffffffffa00927cc>] btrfs_item_size+0x3c/0x90 [btrfs]
[ 4673.544081] [<ffffffffa0067b92>] __btrfs_free_extent+0x222/0x650 [btrfs]
[ 4673.544081] [<ffffffffa009b0a8>] ? extent_write_cache_pages.isra.21.constprop.31+0x108/0x3e0 [btrfs]
[ 4673.544081] [<ffffffffa00680d4>] run_delayed_tree_ref+0x114/0x1a0 [btrfs]
[ 4673.544081] [<ffffffff8115e6df>] ? kmem_cache_free+0x2f/0x110
[ 4673.544081] [<ffffffffa006bd3e>] run_one_delayed_ref+0xae/0xf0 [btrfs]
[ 4673.544081] [<ffffffffa006be54>] run_clustered_refs+0xd4/0x240 [btrfs]
[ 4673.544081] [<ffffffffa006c08a>] btrfs_run_delayed_refs+0xca/0x220 [btrfs]
[ 4673.544081] [<ffffffffa0096246>] ? btrfs_run_ordered_operations+0x1d6/0x1f0 [btrfs]
[ 4673.544081] [<ffffffffa007c113>] btrfs_commit_transaction+0x93/0x840 [btrfs]
[ 4673.544081] [<ffffffff810892b0>] ? add_wait_queue+0x60/0x60
[ 4673.544081] [<ffffffffa008ab77>] btrfs_sync_file+0x187/0x1f0 [btrfs]
[ 4673.544081] [<ffffffff811a2666>] do_fsync+0x56/0x80
[ 4673.544081] [<ffffffff811a29b3>] sys_fdatasync+0x13/0x20
[ 4673.544081] [<ffffffff8165a042>] system_call_fastpath+0x16/0x1b
[ 4673.544081] Code: 83 c0 01 48 89 85 78 ff ff ff e9 c0 fc ff ff 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 55 41 54 53 48 83 ec 08 66 66 66 66 90 <4c> 8b 27 4d 89 cd 48 89 cb 41 81 e4 ff 0f 00 00 4a 8d 04 26 4c
[ 4673.544081] RIP [<ffffffffa009d022>] map_private_extent_buffer+0x12/0x150 [btrfs]
[ 4673.544081] RSP <ffff88001ce2fb28>
[ 4673.544081] CR2: 0000000000000000
[ 4673.574872] ---[ end trace dfc590b622064b17 ]---

[Colin Ian King (colin-king)]

Just to add, this is repeatable each time. I've throughly exercised it on different real H/W and not been able to trip it on H/W, just inside a virtual machine.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 965514

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Colin Ian King (colin-king)]

Can't easily able to run apport-collect on this virtualized instance.

[Brad Figg (brad-figg)]

Thank you for taking the time to file a bug report on this issue.

However, given the number of bugs that the Kernel Team receives during any development cycle it is impossible for us to review them all. Therefore, we occasionally resort to using automated bots to request further testing. This is such a request.

We have noted that there is a newer version of the development kernel than the one you last tested when this issue was found. Please test again with the newer kernel and indicate in the bug if this issue still exists or not.

You can update to the latest development kernel by simply running the following commands in a terminal window:

    sudo apt-get update
    sudo apt-get dist-upgrade

If the bug still exists, change the bug status from Incomplete to Confirmed. If the bug no longer exists, change the bug status from Incomplete to Fix Released.

If you want this bot to quit automatically requesting kernel tests, add a tag named: bot-stop-nagging.

 Thank you for your help, we really do appreciate it.

[Colin Ian King (colin-king)]

Still happens on Linux server-7362 3.2.0-20-virtual #32-Ubuntu SMP Thu Mar 22 02:42:19 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

[Colin Ian King (colin-king)]

Re-tested in:

Precise: 3.2.0-23.36-generic 3.2.14, still occurs
Quantal: 3.5.0-7.7-generic 3.5.0, can no longer reproduce.