kernel NULL pointer dereference during bcache cache_set_flush call

[James Page]

14.04 + HWE 3.16 kernel.

RAID-0 (3 spindles) + bcache on SSD - ext4 filesystem

I'm unable to access this system - this was taken from the iDRAC serial console.

[ 4.559402] md: raid1 personality registered for level 1
[ 4.568578] scsi 4:0:0:0: Direct-Access ATA PLEXTOR PX-AG256 1.04 PQ: 0 ANSI: 5
[ 4.576976] sd 4:0:0:0: Attached scsi generic sg3 type 0
[ 4.576987] sd 4:0:0:0: [sdd] 500118192 512-byte logical blocks: (256 GB/238 GiB)
[ 4.577334] md/raid1:md0: active with 3 out of 3 mirrors
[ 4.577335] sd 4:0:0:0: [sdd] Write Protect is off
[ 4.577363] md0: detected capacity change from 0 to 1073676288
[ 4.577504] sd 4:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 4.578855] sdd: unknown partition table
[ 4.579183] md0: unknown partition table
[ 4.579384] sd 4:0:0:0: [sdd] Attached SCSI disk
[ 4.599312] random: nonblocking pool is initialized
[ 4.668775] bcache: register_bdev() registered backing device md1
[ 4.695459] bcache: error on eaa3d3fb-19b1-47a1-9184-1ac6e96c8593: unsupported bset version at bucket 3617, block 0, 0 keys, disabling caching
[ 4.708278] bcache: register_cache() registered cache device sdd
[ 4.714340] BUG: unable to handle kernel NULL pointer dereference at 0000000000000018
[ 4.722193] IP: [<ffffffffc01fdba2>] journal_write_work+0x22/0x50 [bcache]
[ 4.729083] PGD c198e6067 PUD c19fca067 PMD 0
[ 4.733572] Oops: 0000 [#1] SMP
[ 4.736832] Modules linked in: bcache raid1 raid0 hid_generic e1000e usbhid ahci ptp psmouse hid libahci pata_acpi pps_core
[ 4.748186] CPU: 23 PID: 385 Comm: kworker/23:1 Not tainted 3.16.0-30-generic #40~14.04.1-Ubuntu
[ 4.756956] Hardware name: Supermicro X8DTT-H/X8DTT-H, BIOS 2.1b 10/28/2011
[ 4.764433] Workqueue: events cache_set_flush [bcache]
[ 4.769585] task: ffff880c1af33d20 ti: ffff880c1983c000 task.ti: ffff880c1983c000
[ 4.777056] RIP: 0010:[<ffffffffc01fdba2>] [<ffffffffc01fdba2>] journal_write_work+0x22/0x50 [bcache]
[ 4.786374] RSP: 0018:ffff880c1983fdc8 EFLAGS: 00010246
[ 4.791678] RAX: 0000000000000000 RBX: ffff880c192acbf8 RCX: 000000007fffffff
[ 4.798801] RDX: 0000000000000000 RSI: 000000007fffffff RDI: ffff880c192acbb0
[ 4.805935] RBP: ffff880c1983fdd8 R08: 0000000000000000 R09: 0000000180220000
[ 4.813068] R10: ffffffff81246186 R11: ffffea0030631e80 R12: ffff880c192acbb0
[ 4.820203] R13: ffff880c192a0ce8 R14: ffff880c192a0de8 R15: ffff880c192a0c50
[ 4.827336] FS: 0000000000000000(0000) GS:ffff880c3fd60000(0000) knlGS:0000000000000000
[ 4.835422] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 4.841159] CR2: 0000000000000018 CR3: 0000000c19d5e000 CR4: 00000000000007e0
[ 4.848282] Stack:
[ 4.850311] ffff880c192acbf8 ffff880c1d9a6098 ffff880c1983fe10 ffffffffc0205088
[ 4.857800] ffff880c1d99d200 ffff880c3fd72c40 ffff880c192a0ce8 0000000000000000
[ 4.865253] 00000000000005c0 ffff880c1983fe58 ffffffff81089ef2 000000003fd72c40
[ 4.872706] Call Trace:
[ 4.875185] [<ffffffffc0205088>] cache_set_flush+0x158/0x1c0 [bcache]
[ 4.881707] [<ffffffff81089ef2>] process_one_work+0x182/0x450
[ 4.887539] [<ffffffff8108a663>] worker_thread+0x123/0x5a0
[ 4.893111] [<ffffffff8108a540>] ? rescuer_thread+0x380/0x380
[ 4.898935] [<ffffffff810912c2>] kthread+0xd2/0xf0
[ 4.903807] [<ffffffff810911f0>] ? kthread_create_on_node+0x1c0/0x1c0
[ 4.910334] [<ffffffff8176ac3c>] ret_from_fork+0x7c/0xb0
[ 4.915732] [<ffffffff810911f0>] ? kthread_create_on_node+0x1c0/0x1c0
[ 4.922249] Code: fe ff ff 66 0f 1f 44 00 00 66 66 66 66 90 55 48 89 e5 41 54 4c 8d 67 b8 53 48 89 fb 4c 89 e7 e8 95 cb 56 c1 48 8b 83 38 01 00 00 <80> 78 18 00 75 10 4c 89 e7 e8 90 ca 56 c1 5b 41 5c 5d c3 0f 1f
[ 4.942252] RIP [<ffffffffc01fdba2>] journal_write_work+0x22/0x50 [bcache]
[ 4.949264] RSP <ffff880c1983fdc8>
[ 4.952749] CR2: 0000000000000018
[ 4.956061] ---[ end trace c2f23bd1c3cc953e ]---

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1425288

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Stefan Bader (smb)]

James, can you please add a little more detail to the history (What was that machine doing on the bcache device? How long was it roughly operational before this issue happed?). Also this appears to be a meta-data corruption that leads to some crash. That potentially now happens on every reboot (yes or no)?

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]