[Gigabyte Z87X-D3H] Kernel crash, appears related to ext4 and ecryptfs

[Maarten Baert]

This problem is very similar, but happened with a different kernel (I was using Arch Linux): I am using firefox, I have encrypted my firefox profile folder with ecryptfs, and the underlying file system is ext4. While I was using firefox, the program became frozen and could not be killed. The 'htop' utility indicated that firefox and some other processes had status 'D', i.e. 'uninterruptible sleep'. My graphical session was still functional, and I could open a terminal and run 'dmesg', but any attempt to save it to the hard disk would freeze bash (uninterruptible sleep, just like firefox). I was able to save the output on a different physical hard disk (i.e. not the one that was used by firefox and ecrypt). After that I tried to reboot, but this caused the kernel to lock up and required a hard reset.

So far I have not been able to reproduce this issue, so I can only give a stack trace for Linux 3.12.6 (from Arch Linux):

[42210.127535] ------------[ cut here ]------------
[42210.127540] kernel BUG at fs/buffer.c:1278!
[42210.127542] invalid opcode: 0000 [#1] PREEMPT SMP
[42210.127544] Modules linked in: md5 ecb ecryptfs cbc sha256_ssse3 sha256_generic encrypted_keys hmac trusted tpm tpm_bios snd_hda_codec_hdmi snd_hda_codec_realtek nls_cp437 vfat fat nvidia(PO) x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd hid_logitech_dj iTCO_wdt iTCO_vendor_support mxm_wmi ip6t_REJECT microcode evdev xt_hl ip6t_rt snd_hda_intel snd_hda_codec drm snd_hwdep snd_pcm nf_conntrack_ipv6 e1000e nf_defrag_ipv6 snd_page_alloc i2c_i801 snd_timer snd psmouse ptp soundcore mei_me i2c_core pps_core shpchp mei lpc_ich ipt_REJECT serio_raw pcspkr xt_LOG xt_recent fan thermal xt_multiport xt_limit xt_tcpudp xt_addrtype wmi video processor button nf_conntrack_ipv4
[42210.127583] nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables ext4 crc16 mbcache jbd2 hid_generic usbhid hid sr_mod cdrom sd_mod ahci libahci libata ehci_pci xhci_hcd ehci_hcd scsi_mod usbcore usb_common
[42210.127600] CPU: 0 PID: 7198 Comm: firefox Tainted: P O 3.12.6-1-ARCH #1
[42210.127601] Hardware name: Gigabyte Technology Co., Ltd. Z87X-D3H/Z87X-D3H-CF, BIOS F7 08/02/2013
[42210.127603] task: ffff8802070c1a10 ti: ffff8800cad36000 task.ti: ffff8800cad36000
[42210.127604] RIP: 0010:[<ffffffff811d6ab8>] [<ffffffff811d6ab8>] __find_get_block+0x288/0x290
[42210.127609] RSP: 0018:ffff8800cad375a0 EFLAGS: 00010046
[42210.127610] RAX: 0000000000000092 RBX: ffff88009a0baaf8 RCX: ffff880222edc800
[42210.127611] RDX: 0000000000001000 RSI: 0000000000d000bb RDI: ffff8800cd0f0700
[42210.127613] RBP: ffff8800cad375f8 R08: 0000000000000003 R09: 0000000000000003
[42210.127614] R10: ffff880222edc800 R11: ffffffffa0256a40 R12: ffff8800cd0f0700
[42210.127615] R13: 0000000000001000 R14: ffff88022eb54400 R15: 0000000000000010
[42210.127616] FS: 0000000000000000(0000) GS:ffff88022ec00000(0000) knlGS:0000000000000000
[42210.127617] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[42210.127618] CR2: 00007fa7e5e00018 CR3: 000000000280d000 CR4: 00000000001407f0
[42210.127619] Stack:
[42210.127621] ffffffff811d6f75 ffff8800cad375f0 ffffffff8109c074 ffff8800cad37fd8
[42210.127624] ffff880222deb000 0000000000d000bb ffff88009a0baaf8 ffff8800cad376b0
[42210.127626] ffff880222edc000 ffff88022eb54400 0000000000000010 ffff8800cad37660
[42210.127628] Call Trace:
[42210.127631] [<ffffffff811d6f75>] ? __getblk+0x25/0x2e0
[42210.127636] [<ffffffff8109c074>] ? dequeue_entity+0x144/0x4d0
[42210.127646] [<ffffffffa0288465>] __ext4_get_inode_loc+0x105/0x440 [ext4]
[42210.127653] [<ffffffffa028affd>] ext4_get_inode_loc+0x1d/0x20 [ext4]
[42210.127659] [<ffffffffa028c866>] ext4_reserve_inode_write+0x26/0xa0 [ext4]
[42210.127665] [<ffffffffa028fed0>] ? ext4_dirty_inode+0x40/0x60 [ext4]
[42210.127671] [<ffffffffa028c91f>] ext4_mark_inode_dirty+0x3f/0x230 [ext4]
[42210.127676] [<ffffffffa028fed0>] ext4_dirty_inode+0x40/0x60 [ext4]
[42210.127681] [<ffffffff811ce24a>] __mark_inode_dirty+0x3a/0x2f0
[42210.127685] [<ffffffff811bec21>] update_time+0x81/0xd0
[42210.127687] [<ffffffff811bee30>] file_update_time+0x80/0xd0
[42210.127691] [<ffffffff8113778c>] __generic_file_aio_write+0x16c/0x3b0
[42210.127695] [<ffffffff8101517f>] ? math_state_restore+0x9f/0x190
[42210.127698] [<ffffffff81137a28>] generic_file_aio_write+0x58/0xa0
[42210.127703] [<ffffffffa0284159>] ext4_file_write+0x99/0x3e0 [ext4]
[42210.127707] [<ffffffffa0753083>] ? __ablk_encrypt+0x43/0x50 [ablk_helper]
[42210.127709] [<ffffffff811524bd>] ? kzfree+0x2d/0x30
[42210.127713] [<ffffffff811a430a>] do_sync_write+0x5a/0x90
[42210.127715] [<ffffffff811a49cd>] vfs_write+0xbd/0x1e0
[42210.127718] [<ffffffff811d0e38>] kernel_write+0x38/0x50
[42210.127721] [<ffffffffa006bce9>] ecryptfs_write_lower+0x29/0x50 [ecryptfs]
[42210.127724] [<ffffffffa006d0d3>] ecryptfs_encrypt_page+0xe3/0x180 [ecryptfs]
[42210.127727] [<ffffffffa006b194>] ecryptfs_writepage+0x14/0x60 [ecryptfs]
[42210.127729] [<ffffffff8113fc73>] __writepage+0x13/0x50
[42210.127731] [<ffffffff8114020b>] write_cache_pages+0x21b/0x510
[42210.127733] [<ffffffff8113fc60>] ? mapping_tagged+0x20/0x20
[42210.127736] [<ffffffff811356cb>] ? filemap_fdatawait_range+0x17b/0x1a0
[42210.127738] [<ffffffff81140540>] generic_writepages+0x40/0x60
[42210.127740] [<ffffffff81141d75>] do_writepages+0x35/0x40
[42210.127743] [<ffffffff81136c49>] __filemap_fdatawrite_range+0x59/0x60
[42210.127745] [<ffffffff81136ccc>] filemap_write_and_wait+0x2c/0x60
[42210.127748] [<ffffffffa006ae12>] ecryptfs_put_lower_file+0x32/0x60 [ecryptfs]
[42210.127750] [<ffffffffa00681f2>] ecryptfs_release+0x12/0x30 [ecryptfs]
[42210.127752] [<ffffffff811a62e1>] __fput+0xa1/0x230
[42210.127754] [<ffffffff811a64be>] ____fput+0xe/0x10
[42210.127758] [<ffffffff81081d2c>] task_work_run+0xbc/0xe0
[42210.127761] [<ffffffff8106507c>] do_exit+0x2bc/0xab0
[42210.127764] [<ffffffff81071a63>] ? __sigqueue_free.part.17+0x33/0x40
[42210.127766] [<ffffffff810658ef>] do_group_exit+0x3f/0xa0
[42210.127769] [<ffffffff8107488d>] get_signal_to_deliver+0x2ad/0x610
[42210.127771] [<ffffffff810144d8>] do_signal+0x48/0x940
[42210.127776] [<ffffffff814eadfd>] ? is_prefetch.isra.14.part.15+0xd2/0x251
[42210.127778] [<ffffffff811c116f>] ? fget_light+0x8f/0xf0
[42210.127781] [<ffffffff81014e38>] do_notify_resume+0x68/0xa0
[42210.127784] [<ffffffff814f5bbc>] retint_signal+0x48/0x8c
[42210.127785] Code: 85 f6 0f 84 4e fe ff ff 4c 89 f7 e8 73 fb ff ff e9 41 fe ff ff 4c 89 e7 e8 66 fb ff ff e9 35 ff ff ff 45 31 f6 eb d1 0f 0b 0f 0b <0f> 0b 66 0f 1f 44 00 00 0f 1f 44 00 00 55 48 89 e5 41 54 53 48
[42210.127808] RIP [<ffffffff811d6ab8>] __find_get_block+0x288/0x290
[42210.127810] RSP <ffff8800cad375a0>
[42210.127812] ---[ end trace 412263861e6006c5 ]---
[42210.127814] Fixing recursive fault but reboot is needed!

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: linux-image-3.11.0-14-generic 3.11.0-14.21
ProcVersionSignature: Ubuntu 3.11.0-14.21-generic 3.11.7
Uname: Linux 3.11.0-14-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu2.2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: maarten 1585 F.... lxpanel
CRDA: Error: [Errno 2] No such file or directory: 'iw'
Date: Fri Jan 3 02:43:32 2014
HibernationDevice: RESUME=UUID=bc17e234-da75-457f-b17c-22d9c0e27dd8
InstallationDate: Installed on 2013-12-28 (6 days ago)
InstallationMedia: Lubuntu 13.10 "Saucy Salamander" - Release amd64 (20131016.1)
IwConfig:
 eth0 no wireless extensions.

 lo no wireless extensions.
MachineType: Gigabyte Technology Co., Ltd. Z87X-D3H
MarkForUpload: True
ProcFB: 0 EFI VGA
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.11.0-14-generic.efi.signed root=UUID=5a8ae1fc-91bf-4ce0-8dea-a519976fd56b ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.11.0-14-generic N/A
 linux-backports-modules-3.11.0-14-generic N/A
 linux-firmware 1.116
RfKill:

SourcePackage: linux
StagingDrivers: zram
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 08/02/2013
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: F7
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: Z87X-D3H-CF
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrF7:bd08/02/2013:svnGigabyteTechnologyCo.,Ltd.:pnZ87X-D3H:pvrTobefilledbyO.E.M.:rvnGigabyteTechnologyCo.,Ltd.:rnZ87X-D3H-CF:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvrToBeFilledByO.E.M.:
dmi.product.name: Z87X-D3H
dmi.product.version: To be filled by O.E.M.
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

[penalvch (penalvch)]

Maarten Baert, thank you for reporting this and helping make Ubuntu better. Just to confirm, this is not reproducible in Ubuntu at all, but only in Arch, correct?

[Maarten Baert (maarten-baert)]

No, I meant I have no idea how to reproduce it at all, either on Arch or Ubuntu. It happened for the first time a few days ago, and again 5 minutes ago (also on Arch, because my Ubuntu install isn't using ecryptfs at the moment).

I will try to set up ecryptfs on my Ubuntu install and try to trigger the bug. I've also moved my firefox profile outside ecryptfs on my Arch install to see if this is really related to ecryptfs or just firefox.

Here's the stack trace from the new crash. It's nearly identical: firefox, ecryptfs and ext4. The symptoms were exactly the same except that this time firefox was killed instead of frozen (but other applications would still freeze when writing to the disk).

[ 1525.865720] ------------[ cut here ]------------
[ 1525.865725] kernel BUG at fs/buffer.c:1278!
[ 1525.865726] invalid opcode: 0000 [#1] PREEMPT SMP
[ 1525.865728] Modules linked in: xfs nilfs2 jfs btrfs raid6_pq libcrc32c xor fuse md5 ecb ecryptfs cbc sha256_ssse3 sha256_generic encrypted_keys hmac trusted tpm tpm_bios snd_hda_codec_hdmi snd_hda_codec_realtek nls_cp437 vfat fat nvidia(PO) x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd mxm_wmi iTCO_wdt iTCO_vendor_support microcode hid_logitech_dj psmouse serio_raw evdev ip6t_REJECT pcspkr i2c_i801 xt_hl ip6t_rt drm nf_conntrack_ipv6 nf_defrag_ipv6 i2c_core ipt_REJECT xt_LOG snd_hda_intel snd_hda_codec fan xt_recent snd_hwdep snd_pcm snd_page_alloc xt_multiport e1000e snd_timer snd mei_me soundcore xt_limit xt_tcpudp ptp shpchp mei xt_addrtype pps_core wmi
[ 1525.865755] processor lpc_ich video button thermal nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables ext4 crc16 mbcache jbd2 hid_generic usbhid hid sr_mod cdrom sd_mod ahci libahci ehci_pci xhci_hcd libata ehci_hcd scsi_mod usbcore usb_common
[ 1525.865768] CPU: 0 PID: 839 Comm: firefox Tainted: P O 3.12.6-1-ARCH #1
[ 1525.865769] Hardware name: Gigabyte Technology Co., Ltd. Z87X-D3H/Z87X-D3H-CF, BIOS F7 08/02/2013
[ 1525.865770] task: ffff88007fe64e30 ti: ffff8800c6862000 task.ti: ffff8800c6862000
[ 1525.865771] RIP: 0010:[<ffffffff811d6ab8>] [<ffffffff811d6ab8>] __find_get_block+0x288/0x290
[ 1525.865775] RSP: 0018:ffff8800c68635a0 EFLAGS: 00010046
[ 1525.865776] RAX: 0000000000000092 RBX: ffff8802156393e8 RCX: ffff88007ff73800
[ 1525.865777] RDX: 0000000000001000 RSI: 0000000000d00068 RDI: ffff880225c28380
[ 1525.865777] RBP: ffff8800c68635f8 R08: 0000000000000003 R09: 0000000000000003
[ 1525.865778] R10: ffff88007ff73800 R11: 0000000000000001 R12: ffff880225c28380
[ 1525.865779] R13: 0000000000001000 R14: ffff88022eb20400 R15: 0000000000000010
[ 1525.865780] FS: 00007f1d7364b740(0000) GS:ffff88022ec00000(0000) knlGS:0000000000000000
[ 1525.865781] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1525.865781] CR2: 00007f2729800018 CR3: 000000000280d000 CR4: 00000000001407f0
[ 1525.865782] DR0: 0000000000000000 DR1: 0000000000000000 DR...

[penalvch (penalvch)]

Maarten Baert, thank you for your comments. In your Ubuntu test, could you please confirm this issue exists with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ . If the issue remains, please just make a comment to this.

If reproducible, could you also please test the latest upstream kernel available (not the daily folder) following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this bug is fixed in the mainline kernel, please add the following tags:
kernel-fixed-upstream
kernel-fixed-upstream-VERSION-NUMBER

where VERSION-NUMBER is the version number of the kernel you tested. For example:
kernel-fixed-upstream-v3.13-rc6

This can be done by clicking on the yellow circle with a black pencil icon next to the word Tags located at the bottom of the bug description. As well, please remove the tag:
needs-upstream-testing

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-VERSION-NUMBER

As well, please remove the tag:
needs-upstream-testing

Once testing of the upstream kernel is complete, please mark this bug's Status as Confirmed. Please let us know your results. Thank you for your understanding.

[Maarten Baert (maarten-baert)]

My attempts to reproduce the bug on purpose have failed so far, bug I did find a very simple way to reproduce a similar bug that may have the same cause:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1265841

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]