kernel crash and hung up while umounting xfs

[york]

OS: Ubuntu 18.04.1
Using XFS with Ceph rdb and mount by OverlayFS, sometimes system hung up with kernel crash like below:

2022-07-27 16:17
Jul 27 15:21:43 k8s-node-b01-17-05 kernel: [25666453.856553] VFS: Busy inodes after unmount of overlay. Self-destruct in 5 seconds. Have a nice day...
Jul 27 15:22:01 k8s-node-b01-17-05 kernel: [25666471.122435] XFS (rbd59): Unmounting Filesystem
Jul 27 15:22:25 k8s-node-b01-17-05 kernel: [25666495.392431] BUG: Dentry 00000000d683c981{i=38c31f07,n=README_zh.md} still in use (1) [unmount of xfs rbd23]

[Chris Guiver (guiverc)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:

apport-collect 1983741

When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1983741

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[hyz (hyz0906-h)]

After run the command, and set kernel.core_pattern to /var/crash/core-**, we cannot find any log,and there is no crash log catched in the dir /var/crash/

[hyz (hyz0906-h)]

And sometimes the system reboots with kernel panic:
call stack:
vfs_read
ksys_read
fput
...
...
RIP: 0033:0x4934bb

RIP: 0100:string_check

[hyz (hyz0906-h)]

kernel panic

[hyz (hyz0906-h)]

Sep 21 10:53:21 b02-08-07 kernel: [6181670.539888] VFS: Busy inodes after unmount of overlay. Self-destruct in 5 seconds. Have a nice day...
Sep 21 10:53:28 b02-08-07 kernel: [6181677.010648] BUG: Dentry 00000000e26df9ea{i=31d3dce7,n=drivers} still in use (1) [unmount of xfs rbd13]
Sep 21 10:53:28 b02-08-07 kernel: [6181677.010947] ------------[ cut here ]------------
Sep 21 10:53:28 b02-08-07 kernel: [6181677.010972] WARNING: CPU: 90 PID: 827294 at /build/linux-hwe-UUUsac/linux-hwe-5.3.0/fs/dcache.c:1596 umount_check+0x69/0x80
Sep 21 10:53:28 b02-08-07 kernel: [6181677.010974] Modules linked in: xfs xt_set ipt_rpfilter xt_multiport iptable_raw ip_set_hash_ip ip_set_hash_net ip_set veth ip6table_nat ip6_tables rbd libceph xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 nbd ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs iptable_mangle xt_comment xt_mark bluetooth ecdh_generic ecc nfsv3 nfs_acl xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_nat br_netfilter bridge rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace aufs fscache overlay xt_tcpudp iptable_filter bpfilter openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 8021q garp mrp stp llc bonding intel_rapl_msr intel_rapl_common isst_if_common skx_edac nfit x86_pkg_temp_thermal nls_iso8859_1 coretemp raid10 dm_raid kvm_intel raid456 async_raid6_recov async_memcpy kvm async_pq async_xor async_tx irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 crypto_simd cryptd glue_helper ipmi_si ipmi_devintf
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011028] ipmi_msghandler intel_cstate intel_rapl_perf ioatdma mei_me dca mei joydev input_leds lpc_ich mac_hid acpi_power_meter sunrpc sch_fq_codel ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq libcrc32c megaraid_sas mlx5_ib ses enclosure ib_uverbs ib_core hid_generic mlx5_core tls mlxfw i40e smartpqi hinic(OE) scsi_transport_sas nvme usbhid nvme_core ahci hid libahci
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011059] CPU: 90 PID: 827294 Comm: umount Tainted: G W OE 5.3.0-45-generic #37~18.04.1-Ubuntu
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011061] Hardware name: Huawei 2488H V5/BC62MBHA0, BIOS 7.55 04/21/2020
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011063] RIP: 0010:umount_check+0x69/0x80
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011064] Code: 08 48 8b 46 30 48 85 c0 74 27 48 8b 50 40 51 48 c7 c7 50 78 d4 b1 48 89 f1 e8 a6 11 e2 ff 48 c7 c7 a0 3e d0 b1 e8 9a 11 e2 ff <0f> 0b 58 31 c0 c9 c3 31 d2 eb d9 66 90 66 2e 0f 1f 84 00 00 00 00
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011066] RSP: 0018:ffffc008b9493d50 EFLAGS: 00010282
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011067] RAX: 0000000000000024 RBX: ffff98fb7c12a390 RCX: 0000000000000006
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011068] RDX: 0000000000000000 RSI: 0000000000000082 RDI: ffff99317f497440
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011069] RBP: ffffc008b9493d58 R08: 00000000000017f3 R09: 0000000000000004
Sep 21 10:53:28 b02-08-07 kernel: [6181677.011069] R10: ffffc008b9493c68 R11: 0000000000000001 R12: ffff98fb7c12a300
Sep 21 10:53:28 b02-08-07 kernel: [6181677.01...

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]