kernel BUG at fs/cachefiles/rdwr.c:717! - system completly locks up and requires hard reset

[Marcel Noe]

This is a virtual machine with PCIe passthrough and home filesystem mounted from host over NFS. The system also runs FSC to cache the NFS. The system occurs since the last update - ubuntu live patch is enabled.

The system boots up and launches lightdm. When logging in as normal user, the system immediately locks up. This also happens when logging in over ssh as normal user having his home on the NFS share.

When logging in as root over ssh, and running dmesg -w, I see the following message:

[ 46.358431] CacheFiles:
[ 46.358436] CacheFiles: Assertion failed
[ 46.358545] ------------[ cut here ]------------
[ 46.358547] kernel BUG at fs/cachefiles/rdwr.c:717!
[ 46.358568] invalid opcode: 0000 [#1] SMP PTI
[ 46.358572] CPU: 0 PID: 3756 Comm: sshd Tainted: P O 5.8.0-40-generic #45~20.04.1-Ubuntu
[ 46.358574] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
[ 46.358594] RIP: 0010:cachefiles_read_or_alloc_pages.cold+0x32/0x4e [cachefiles]
[ 46.358598] Code: a6 ef 02 e9 48 c7 c7 48 e8 2e c2 e8 9a ef 02 e9 0f 0b 48 c7 c7 70 db 2e c2 e8 8c ef 02 e9 48 c7 c7 48 e8 2e c2 e8 80 ef 02 e9 <0f> 0b 48 c7 c7 70 db 2e c2 e8 72 ef 02 e9 48 c7 c7 48 e8 2e c2 e8
[ 46.358600] RSP: 0018:ffffb059010d7890 EFLAGS: 00010246
[ 46.358603] RAX: 000000000000001c RBX: ffff8b47cf46a770 RCX: 0000000000000027
[ 46.358605] RDX: 0000000000000000 RSI: 0000000000000086 RDI: ffff8b482fa18cd8
[ 46.358607] RBP: ffffb059010d7980 R08: ffff8b482fa18cd0 R09: ffffffffacdcf730
[ 46.358609] R10: ffffffffacdb4988 R11: ffffb059010d7710 R12: ffff8b47cb0ed8c0
[ 46.358610] R13: 0000000000000000 R14: ffffb059010d7b98 R15: ffffb059010d7a24
[ 46.358613] FS: 00007f351cd26f40(0000) GS:ffff8b482fa00000(0000) knlGS:0000000000000000
[ 46.358615] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 46.358617] CR2: 0000557fbc630ee0 CR3: 000000044cbc6005 CR4: 00000000001606f0
[ 46.358622] Call Trace:
[ 46.358642] ? update_load_avg+0x7c/0x640
[ 46.358647] ? native_smp_send_reschedule+0x2a/0x40
[ 46.358650] ? wake_up_bit+0x42/0x50
[ 46.358674] ? fscache_run_op.isra.0+0x5d/0xc0 [fscache]
[ 46.358683] __fscache_read_or_alloc_pages+0x234/0x2f0 [fscache]
[ 46.358704] __nfs_readpages_from_fscache+0x65/0x150 [nfs]
[ 46.358720] nfs_readpages+0xc2/0x1b0 [nfs]
[ 46.358724] ? get_page_from_freelist+0x1b5/0x370
[ 46.358728] ? __alloc_pages_nodemask+0x161/0x2f0
[ 46.358732] read_pages+0x194/0x280
[ 46.358736] page_cache_readahead_unbounded+0x147/0x200
[ 46.358741] __do_page_cache_readahead+0x35/0x40
[ 46.358744] ondemand_readahead+0x148/0x2a0
[ 46.358747] page_cache_sync_readahead+0x78/0xc0
[ 46.358752] generic_file_buffered_read+0x598/0xc50
[ 46.358763] ? __mod_lruvec_state+0x41/0xf0
[ 46.358768] generic_file_read_iter+0xdc/0x140
[ 46.358773] ? _cond_resched+0x19/0x30
[ 46.358786] ? nfs_revalidate_mapping+0xa1/0x290 [nfs]
[ 46.358798] nfs_file_read+0x74/0xb0 [nfs]
[ 46.358803] new_sync_read+0x10c/0x1a0
[ 46.358808] vfs_read+0x161/0x190
[ 46.358811] ksys_read+0x67/0xe0
[ 46.358815] __x64_sys_read+0x1a/0x20
[ 46.358820] do_syscall_64+0x49/0xc0
[ 46.358823] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 46.358826] RIP: 0033:0x7f351d116142
[ 46.358829] Code: c0 e9 c2 fe ff ff 50 48 8d 3d 3a ca 0a 00 e8 f5 19 02 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
[ 46.358831] RSP: 002b:00007ffc63e16e78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 46.358834] RAX: ffffffffffffffda RBX: 0000557fbc621ee0 RCX: 00007f351d116142
[ 46.358836] RDX: 0000000000002000 RSI: 0000557fbc62eed0 RDI: 0000000000000005
[ 46.358837] RBP: 00007f351d1f24a0 R08: 0000000000000005 R09: 00007f351d1f0c20
[ 46.358839] R10: 0000557fbc5e4010 R11: 0000000000000246 R12: 00007f351cd26f40
[ 46.358840] R13: 00007f351d1f18a0 R14: 0000000000000d68 R15: 0000000000000d68
[ 46.358843] Modules linked in: rfcomm cachefiles xt_multiport ip6table_filter ip6_tables xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c bpfilter br_netfilter bridge stp llc rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd aufs overlay grace cmac algif_hash algif_skcipher af_alg bnep snd_hda_codec_hdmi nvidia_uvm(O) nvidia_drm(PO) nvidia_modeset(PO) nls_iso8859_1 intel_rapl_msr intel_rapl_common snd_hda_codec_generic ledtrig_audio crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd snd_hda_intel cryptd glue_helper snd_intel_dspcfg btusb snd_hda_codec btrtl snd_usb_audio btbcm btintel snd_hda_core snd_usbmidi_lib snd_hwdep bluetooth mc snd_seq_midi snd_seq_midi_event ecdh_generic rapl nvidia(PO) razerkbd(O) snd_pcm ecc joydev snd_rawmidi input_leds snd_seq serio_raw efi_pstore snd_seq_device snd_timer qxl snd drm_ttm_helper ttm soundcore drm_kms_helper cec
[ 46.358904] rc_core fb_sys_fops syscopyarea sysfillrect qemu_fw_cfg sysimgblt mac_hid sch_fq_codel 9pnet_virtio 9p 9pnet fscache parport_pc ppdev lp parport drm sunrpc virtio_rng ip_tables x_tables autofs4 hid_generic usbhid hid crc32_pclmul ahci i2c_i801 psmouse libahci virtio_net lpc_ich xhci_pci i2c_smbus xhci_pci_renesas net_failover virtio_blk failover
[ 46.358929] ---[ end trace 3971cc4187bb6073 ]---
[ 47.231347] RIP: 0010:cachefiles_read_or_alloc_pages.cold+0x32/0x4e [cachefiles]
[ 47.231368] Code: a6 ef 02 e9 48 c7 c7 48 e8 2e c2 e8 9a ef 02 e9 0f 0b 48 c7 c7 70 db 2e c2 e8 8c ef 02 e9 48 c7 c7 48 e8 2e c2 e8 80 ef 02 e9 <0f> 0b 48 c7 c7 70 db 2e c2 e8 72 ef 02 e9 48 c7 c7 48 e8 2e c2 e8
[ 47.231369] RSP: 0018:ffffb059010d7890 EFLAGS: 00010246
[ 47.231374] RAX: 000000000000001c RBX: ffff8b47cf46a770 RCX: 0000000000000027
[ 47.231375] RDX: 0000000000000000 RSI: 0000000000000086 RDI: ffff8b482fa18cd8
[ 47.231376] RBP: ffffb059010d7980 R08: ffff8b482fa18cd0 R09: ffffffffacdcf730
[ 47.231377] R10: ffffffffacdb4988 R11: ffffb059010d7710 R12: ffff8b47cb0ed8c0
[ 47.231378] R13: 0000000000000000 R14: ffffb059010d7b98 R15: ffffb059010d7a24
[ 47.231379] FS: 00007f351cd26f40(0000) GS:ffff8b482fa00000(0000) knlGS:0000000000000000
[ 47.231380] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 47.231381] CR2: 0000557fbc630ee0 CR3: 000000044cbc6005 CR4: 00000000001606f0

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.8.0-40-generic 5.8.0-40.45~20.04.1
ProcVersionSignature: Ubuntu 5.8.0-40.45~20.04.1-generic 5.8.18
Uname: Linux 5.8.0-40-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu27.14
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed Jan 27 11:22:15 2021
InstallationDate: Installed on 2020-07-10 (200 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Release amd64 (20200423)
SourcePackage: linux-signed-hwe-5.8
UpgradeStatus: No upgrade log present (probably fresh install)

[Marcel Noe (marcel-marcel-noe)]

When booting to 5.4.0-59-generic #65-Ubuntu SMP everything works fine. I therefore guess that this bug was newly introduced between 5.4.0-59 and 5.8.0-40.

[Jure Koren (korenjure)]

I think this could be related to:
https://bugzilla.kernel.org/show_bug.cgi?id=208883