Apr 29 17:12:50 cluster1 systemd[1]: Started Hostname Service. Apr 29 17:12:50 cluster1 dbus-daemon[502]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkit.service' requested by ':1.5' (uid=0 pid=575 comm="/lib/systemd/systemd-hostnamed " label="unconfined") Apr 29 17:12:50 cluster1 systemd[1]: Starting Authorization Manager... Apr 29 17:12:51 cluster1 systemd[1]: Finished NFS server and services. Apr 29 17:12:51 cluster1 systemd[1]: Starting Notify NFS peers of a restart... Apr 29 17:12:51 cluster1 sm-notify[591]: Version 2.6.1 starting Apr 29 17:12:51 cluster1 systemd[1]: Started Notify NFS peers of a restart. Apr 29 17:12:51 cluster1 networkd-dispatcher[509]: No valid path found for iwconfig Apr 29 17:12:51 cluster1 networkd-dispatcher[509]: No valid path found for iw Apr 29 17:12:51 cluster1 systemd[1]: Started Dispatcher daemon for systemd-networkd. Apr 29 17:12:51 cluster1 polkitd[582]: started daemon version 0.105 using authority implementation `local' version `0.105' Apr 29 17:12:51 cluster1 dbus-daemon[502]: [system] Successfully activated service 'org.freedesktop.PolicyKit1' Apr 29 17:12:51 cluster1 systemd[1]: Started Authorization Manager. Apr 29 17:12:52 cluster1 systemd-hostnamed[575]: Hostname set to (static) Apr 29 17:12:52 cluster1 systemd[1]: Created slice User Slice of UID 1000. Apr 29 17:12:52 cluster1 systemd[1]: Starting User Runtime Directory /run/user/1000... Apr 29 17:12:52 cluster1 systemd[1]: Finished User Runtime Directory /run/user/1000. Apr 29 17:12:52 cluster1 systemd[1]: Starting User Manager for UID 1000... Apr 29 17:12:53 cluster1 snapd[514]: overlord.go:271: Acquiring state lock file Apr 29 17:12:53 cluster1 snapd[514]: overlord.go:276: Acquired state lock file Apr 29 17:12:55 cluster1 snapd[514]: daemon.go:247: started snapd/2.62 (series 16; classic) ubuntu/22.04 (amd64) linux/6.5.0-28-generic. Apr 29 17:12:55 cluster1 kernel: [ 35.467295] loop9: detected capacity change from 0 to 8 Apr 29 17:12:55 cluster1 snapd[514]: daemon.go:340: adjusting startup timeout by 1m15s (pessimistic estimate of 30s plus 5s per snap) Apr 29 17:12:55 cluster1 systemd[1]: tmp-syscheck\x2dmountpoint\x2d1475474335.mount: Deactivated successfully. Apr 29 17:12:55 cluster1 systemd[636]: Queued start job for default target Main User Target. Apr 29 17:12:55 cluster1 systemd[636]: Created slice User Application Slice. Apr 29 17:12:55 cluster1 systemd[636]: Reached target Paths. Apr 29 17:12:55 cluster1 systemd[636]: Reached target Timers. Apr 29 17:12:55 cluster1 systemd[636]: Starting D-Bus User Message Bus Socket... Apr 29 17:12:55 cluster1 systemd[636]: Listening on GnuPG network certificate management daemon. Apr 29 17:12:55 cluster1 systemd[636]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers). Apr 29 17:12:55 cluster1 systemd[636]: Listening on GnuPG cryptographic agent and passphrase cache (restricted). Apr 29 17:12:55 cluster1 systemd[636]: Listening on GnuPG cryptographic agent (ssh-agent emulation). Apr 29 17:12:55 cluster1 systemd[636]: Listening on GnuPG cryptographic agent and passphrase cache. Apr 29 17:12:55 cluster1 systemd[636]: Listening on debconf communication socket. Apr 29 17:12:55 cluster1 systemd[636]: Listening on REST API socket for snapd user session agent. Apr 29 17:12:55 cluster1 systemd[636]: Listening on D-Bus User Message Bus Socket. Apr 29 17:12:55 cluster1 systemd[636]: Reached target Sockets. Apr 29 17:12:55 cluster1 systemd[636]: Reached target Basic System. Apr 29 17:12:55 cluster1 systemd[636]: Reached target Main User Target. Apr 29 17:12:55 cluster1 systemd[636]: Startup finished in 2.200s. Apr 29 17:12:55 cluster1 systemd[1]: Started User Manager for UID 1000. Apr 29 17:12:55 cluster1 systemd[1]: Started Session c1 of User wesley. Apr 29 17:12:57 cluster1 snapd[514]: backends.go:58: AppArmor status: apparmor is enabled and all features are available (using snapd provided apparmor_parser) Apr 29 17:12:57 cluster1 systemd[1]: Started Snap Daemon. Apr 29 17:12:57 cluster1 dbus-daemon[502]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.13' (uid=0 pid=514 comm="/usr/lib/snapd/snapd " label="unconfined") Apr 29 17:12:57 cluster1 systemd[1]: Starting Time & Date Service... Apr 29 17:12:58 cluster1 dbus-daemon[502]: [system] Successfully activated service 'org.freedesktop.timedate1' Apr 29 17:12:58 cluster1 systemd[1]: Started Time & Date Service. Apr 29 17:12:58 cluster1 systemd[1]: dmesg.service: Deactivated successfully. Apr 29 17:12:58 cluster1 systemd[1]: Finished Wait until snapd is fully seeded. Apr 29 17:12:58 cluster1 systemd[1]: Starting Apply the settings specified in cloud-config... Apr 29 17:12:58 cluster1 systemd[1]: Condition check resulted in Auto import assertions from block devices being skipped. Apr 29 17:12:59 cluster1 cloud-init[712]: Cloud-init v. 24.1.3-0ubuntu1~22.04.1 running 'modules:config' at Mon, 29 Apr 2024 22:12:59 +0000. Up 39.18 seconds. Apr 29 17:12:59 cluster1 systemd[1]: Finished Apply the settings specified in cloud-config. Apr 29 17:13:00 cluster1 nmbd[510]: [2024/04/29 17:12:59.872061, 0] ../../source3/nmbd/nmbd.c:901(main) Apr 29 17:13:00 cluster1 nmbd[510]: nmbd version 4.15.13-Ubuntu started. Apr 29 17:13:00 cluster1 nmbd[510]: Copyright Andrew Tridgell and the Samba Team 1992-2021 Apr 29 17:13:00 cluster1 systemd[1]: Started Samba NMB Daemon. Apr 29 17:13:00 cluster1 systemd[1]: Starting Samba SMB Daemon... Apr 29 17:13:00 cluster1 systemd[636]: Started D-Bus User Message Bus. Apr 29 17:13:00 cluster1 dbus-daemon[740]: [session uid=1000 pid=740] AppArmor D-Bus mediation is enabled Apr 29 17:13:01 cluster1 systemd[636]: Started snap.go.go-81631c53-a1bd-4726-90e8-0f9cae27b7fb.scope. Apr 29 17:13:02 cluster1 smbd[719]: [2024/04/29 17:13:02.563773, 0] ../../source3/smbd/server.c:1734(main) Apr 29 17:13:02 cluster1 smbd[719]: smbd version 4.15.13-Ubuntu started. Apr 29 17:13:02 cluster1 smbd[719]: Copyright Andrew Tridgell and the Samba Team 1992-2021 Apr 29 17:13:02 cluster1 systemd[1]: e2scrub_reap.service: Deactivated successfully. Apr 29 17:13:02 cluster1 systemd[1]: Finished Remove Stale Online ext4 Metadata Check Snapshots. Apr 29 17:13:03 cluster1 systemd[636]: Started snap.go.go-3c391636-0b9c-45da-b9ef-5964fed83b8d.scope. Apr 29 17:13:03 cluster1 systemd[636]: Started snap.go.go-31119982-bb05-432c-a3d8-c6426a4169ba.scope. Apr 29 17:13:03 cluster1 systemd[636]: Started snap.go.go-57aec246-c080-4a1c-8678-23fe9f81217c.scope. Apr 29 17:13:03 cluster1 systemd[636]: Started snap.go.go-0031c3f3-2109-4fa6-8da7-62a3af4feb00.scope. Apr 29 17:13:03 cluster1 systemd[636]: Started snap.go.go-269b28ab-6e63-4c14-9d40-a705e1a3ed8a.scope. Apr 29 17:13:04 cluster1 systemd[1]: Started Samba SMB Daemon. Apr 29 17:13:04 cluster1 systemd[1]: Reached target Multi-User System. Apr 29 17:13:04 cluster1 systemd[1]: Reached target Graphical Interface. Apr 29 17:13:04 cluster1 systemd[1]: Starting Execute cloud user/final scripts... Apr 29 17:13:04 cluster1 systemd[1]: Starting Record Runlevel Change in UTMP... Apr 29 17:13:04 cluster1 systemd[1]: systemd-update-utmp-runlevel.service: Deactivated successfully. Apr 29 17:13:04 cluster1 systemd[1]: Finished Record Runlevel Change in UTMP. Apr 29 17:13:04 cluster1 systemd[636]: Started snap.go.go-2f0ea3cb-84f7-4135-8d18-5b06b98840b9.scope. Apr 29 17:13:04 cluster1 systemd[636]: Started snap.go.go-0167c87c-44b6-42c8-8936-66e9594912fa.scope. Apr 29 17:13:04 cluster1 systemd[636]: Started snap.go.go-d69cf79d-87b9-472e-9132-064a767cf847.scope. Apr 29 17:13:04 cluster1 cloud-init[990]: Cloud-init v. 24.1.3-0ubuntu1~22.04.1 running 'modules:final' at Mon, 29 Apr 2024 22:13:04 +0000. Up 44.54 seconds. Apr 29 17:13:04 cluster1 cloud-init[990]: Cloud-init v. 24.1.3-0ubuntu1~22.04.1 finished at Mon, 29 Apr 2024 22:13:04 +0000. Datasource DataSourceLXD. Up 44.71 seconds Apr 29 17:13:04 cluster1 systemd[1]: Finished Execute cloud user/final scripts. Apr 29 17:13:04 cluster1 systemd[1]: Reached target Cloud-init target. Apr 29 17:13:04 cluster1 systemd[1]: Startup finished in 5.964s (kernel) + 38.817s (userspace) = 44.782s. Apr 29 17:13:09 cluster1 systemd[1]: systemd-fsckd.service: Deactivated successfully. Apr 29 17:13:15 cluster1 systemd-timesyncd[416]: Initial synchronization to time server 185.125.190.58:123 (ntp.ubuntu.com). Apr 29 17:13:22 cluster1 systemd[1]: systemd-hostnamed.service: Deactivated successfully. Apr 29 17:13:23 cluster1 nmbd[510]: [2024/04/29 17:13:23.067026, 0] ../../source3/nmbd/nmbd_become_lmb.c:398(become_local_master_stage2) Apr 29 17:13:23 cluster1 nmbd[510]: ***** Apr 29 17:13:23 cluster1 nmbd[510]: Apr 29 17:13:23 cluster1 nmbd[510]: Samba name server CLUSTER1 is now a local master browser for workgroup WORKGROUP on subnet 192.168.3.150 Apr 29 17:13:23 cluster1 nmbd[510]: Apr 29 17:13:23 cluster1 nmbd[510]: ***** Apr 29 17:13:23 cluster1 kernel: [ 63.846062] spl: loading out-of-tree module taints kernel. Apr 29 17:13:24 cluster1 kernel: [ 64.173320] zfs: module license 'CDDL' taints kernel. Apr 29 17:13:24 cluster1 kernel: [ 64.173330] Disabling lock debugging due to kernel taint Apr 29 17:13:24 cluster1 kernel: [ 64.173355] zfs: module license taints kernel. Apr 29 17:13:26 cluster1 kernel: [ 66.219837] ZFS: Loaded module v2.2.0-0ubuntu1~23.10.2, ZFS pool version 5000, ZFS filesystem version 5 Apr 29 17:13:26 cluster1 systemd-udevd[1045]: Using default interface naming scheme 'v249'. Apr 29 17:13:26 cluster1 kernel: [ 66.654861] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. Apr 29 17:13:26 cluster1 networkd-dispatcher[509]: WARNING:Unknown index 3 seen, reloading interface list Apr 29 17:13:26 cluster1 networkd-dispatcher[509]: WARNING:Unknown index 4 seen, reloading interface list Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0-mtu: Link UP Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0-mtu: Gained carrier Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0-mtu: Gained IPv6LL Apr 29 17:13:26 cluster1 kernel: [ 66.709417] lxdfan0: port 1(lxdfan0-mtu) entered blocking state Apr 29 17:13:26 cluster1 kernel: [ 66.709425] lxdfan0: port 1(lxdfan0-mtu) entered disabled state Apr 29 17:13:26 cluster1 kernel: [ 66.709448] lxdfan0-mtu: entered allmulticast mode Apr 29 17:13:26 cluster1 kernel: [ 66.709505] lxdfan0-mtu: entered promiscuous mode Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0: Link UP Apr 29 17:13:26 cluster1 kernel: [ 66.712227] lxdfan0: port 1(lxdfan0-mtu) entered blocking state Apr 29 17:13:26 cluster1 kernel: [ 66.712232] lxdfan0: port 1(lxdfan0-mtu) entered forwarding state Apr 29 17:13:26 cluster1 networkd-dispatcher[509]: WARNING:Unknown index 5 seen, reloading interface list Apr 29 17:13:26 cluster1 kernel: [ 66.960283] lxdfan0: port 2(lxdfan0-fan) entered blocking state Apr 29 17:13:26 cluster1 kernel: [ 66.960291] lxdfan0: port 2(lxdfan0-fan) entered disabled state Apr 29 17:13:26 cluster1 kernel: [ 66.960302] lxdfan0-fan: entered allmulticast mode Apr 29 17:13:26 cluster1 kernel: [ 66.960356] lxdfan0-fan: entered promiscuous mode Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0-fan: Link UP Apr 29 17:13:26 cluster1 systemd-networkd[465]: lxdfan0-fan: Gained carrier Apr 29 17:13:26 cluster1 kernel: [ 66.971904] lxdfan0: port 2(lxdfan0-fan) entered blocking state Apr 29 17:13:26 cluster1 kernel: [ 66.971913] lxdfan0: port 2(lxdfan0-fan) entered forwarding state Apr 29 17:13:26 cluster1 kernel: [ 67.023735] kauditd_printk_skb: 22 callbacks suppressed Apr 29 17:13:26 cluster1 kernel: [ 67.023739] audit: type=1400 audit(1714428806.972:36): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_dnsmasq-lxdfan0_" pid=1082 comm="apparmor_parser" Apr 29 17:13:26 cluster1 kernel: [ 67.039389] audit: type=1400 audit(1714428806.988:37): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_forkdns-lxdfan0_" pid=1084 comm="apparmor_parser" Apr 29 17:13:27 cluster1 dnsmasq[1085]: started, version 2.90 cachesize 150 Apr 29 17:13:27 cluster1 dnsmasq[1085]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset no-nftset auth cryptohash DNSSEC loop-detect inotify dumpfile Apr 29 17:13:27 cluster1 dnsmasq[1085]: LOUD WARNING: listening on 240.150.0.1 may accept requests via interfaces other than lxdfan0 Apr 29 17:13:27 cluster1 dnsmasq[1085]: LOUD WARNING: use --bind-dynamic rather than --bind-interfaces to avoid DNS amplification attacks via these interface(s) Apr 29 17:13:27 cluster1 dnsmasq-dhcp[1085]: DHCP, IP range 240.150.0.2 -- 240.150.0.254, lease time 1h Apr 29 17:13:27 cluster1 dnsmasq-dhcp[1085]: DHCP, sockets bound exclusively to interface lxdfan0 Apr 29 17:13:27 cluster1 dnsmasq[1085]: using nameserver 240.150.0.1#1053 for domain lxd Apr 29 17:13:27 cluster1 dnsmasq[1085]: using nameserver 240.150.0.1#1053 for domain 240.in-addr.arpa Apr 29 17:13:27 cluster1 dnsmasq[1085]: reading /etc/resolv.conf Apr 29 17:13:27 cluster1 dnsmasq[1085]: using nameserver 240.150.0.1#1053 for domain lxd Apr 29 17:13:27 cluster1 dnsmasq[1085]: using nameserver 240.150.0.1#1053 for domain 240.in-addr.arpa Apr 29 17:13:27 cluster1 dnsmasq[1085]: using nameserver 127.0.0.53#53 Apr 29 17:13:27 cluster1 dnsmasq[1085]: read /etc/hosts - 9 names Apr 29 17:13:27 cluster1 dnsmasq-dhcp[1085]: read /var/lib/lxd/networks/lxdfan0/dnsmasq.hosts/test.eth0 Apr 29 17:13:27 cluster1 systemd-networkd[465]: lxdfan0: Gained carrier Apr 29 17:13:28 cluster1 systemd-networkd[465]: lxdfan0-fan: Gained IPv6LL Apr 29 17:13:29 cluster1 systemd[1]: systemd-timedated.service: Deactivated successfully. Apr 29 17:13:44 cluster1 systemd[1]: Started Session c2 of User wesley. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-02e8f214-1bea-4716-983e-746f42df4d88.scope. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-f3c6f1b4-a4df-4ba6-a7ae-99d9b1ee28f8.scope. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-2a2ee7f5-ee5d-47c0-a0b5-8991e46dce92.scope. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-8603a017-8a74-4770-a107-df811d9462b6.scope. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-94369947-3e48-4a72-897b-76b4e12cbc87.scope. Apr 29 17:13:44 cluster1 systemd[636]: Started snap.go.go-781dc070-097b-4bfa-b4a9-5714baba76a5.scope. Apr 29 17:13:45 cluster1 systemd[636]: Started snap.go.go-ba4212b5-ec18-42e2-82e7-dd4ca0c22aa9.scope. Apr 29 17:13:45 cluster1 systemd[636]: Started snap.go.go-59330f28-7cab-4ffb-98bc-065dbd97bf53.scope. Apr 29 17:13:45 cluster1 systemd[636]: Started snap.go.go-53e47fd4-bf60-467a-82a1-79401c04e1f9.scope. Apr 29 17:14:19 cluster1 systemd[1]: Started Session c3 of User wesley. Apr 29 17:14:19 cluster1 systemd[636]: Started snap.go.go-bad67717-51e5-4414-b6c9-82b5f7c6fb07.scope. Apr 29 17:14:19 cluster1 systemd[636]: Started snap.go.go-b9125d4b-b2ce-4948-8b4c-f0168bb42827.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-3d1f2c1b-2fe4-48b3-92c9-8a5a84d371d9.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-0a5c515d-50d7-4565-9649-67c8a9a62379.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-e9ad5455-dd7d-418f-8b86-dee972a20ce0.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-db97eee5-bb11-43b4-8fdf-ae153d0de945.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-808448a2-b460-415b-959a-b6ddf9d30062.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-0aa48456-5b9f-4250-b54f-b452021dd75c.scope. Apr 29 17:14:20 cluster1 systemd[636]: Started snap.go.go-02f09d2d-8c9e-442d-b047-616d46669129.scope. Apr 29 17:14:53 cluster1 dnsmasq[1085]: read /etc/hosts - 9 names Apr 29 17:15:01 cluster1 kernel: [ 161.250951] ------------[ cut here ]------------ Apr 29 17:15:01 cluster1 kernel: [ 161.250957] Voluntary context switch within RCU read-side critical section! Apr 29 17:15:01 cluster1 kernel: [ 161.250990] WARNING: CPU: 2 PID: 510 at kernel/rcu/tree_plugin.h:320 rcu_note_context_switch+0x2a7/0x2f0 Apr 29 17:15:01 cluster1 kernel: [ 161.251003] Modules linked in: nft_masq nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 vxlan ip6_udp_tunnel udp_tunnel dummy bridge stp llc zfs(PO) spl(O) nf_tables libcrc32c nfnetlink vhost_vsock vhost vhost_iotlb binfmt_misc nls_iso8859_1 intel_rapl_msr intel_rapl_common kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul virtio_gpu polyval_clmulni polyval_generic ghash_clmulni_intel sha256_ssse3 sha1_ssse3 virtio_dma_buf aesni_intel vmw_vsock_virtio_transport 9pnet_virtio xhci_pci drm_shmem_helper i2c_i801 ahci 9pnet vmw_vsock_virtio_transport_common xhci_pci_renesas drm_kms_helper libahci crypto_simd joydev virtio_input cryptd lpc_ich virtiofs i2c_smbus vsock psmouse input_leds mac_hid serio_raw rapl qemu_fw_cfg vmgenid nfsd dm_multipath auth_rpcgss scsi_dh_rdac nfs_acl lockd scsi_dh_emc scsi_dh_alua grace sch_fq_codel drm sunrpc efi_pstore virtio_rng ip_tables x_tables autofs4 Apr 29 17:15:01 cluster1 kernel: [ 161.251085] CPU: 2 PID: 510 Comm: nmbd Tainted: P O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:15:01 cluster1 kernel: [ 161.251089] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009)/LXD, BIOS unknown 2/2/2022 Apr 29 17:15:01 cluster1 kernel: [ 161.251091] RIP: 0010:rcu_note_context_switch+0x2a7/0x2f0 Apr 29 17:15:01 cluster1 kernel: [ 161.251095] Code: 08 f0 83 44 24 fc 00 48 89 de 4c 89 f7 e8 d1 af ff ff e9 1e fe ff ff 48 c7 c7 d0 60 56 88 c6 05 e6 27 40 02 01 e8 79 b2 f2 ff <0f> 0b e9 bd fd ff ff a9 ff ff ff 7f 0f 84 75 fe ff ff 65 48 8b 3c Apr 29 17:15:01 cluster1 kernel: [ 161.251098] RSP: 0018:ffffb9cbc11dbbc8 EFLAGS: 00010046 Apr 29 17:15:01 cluster1 kernel: [ 161.251101] RAX: 0000000000000000 RBX: ffff941ef7cb3f80 RCX: 0000000000000000 Apr 29 17:15:01 cluster1 kernel: [ 161.251103] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 Apr 29 17:15:01 cluster1 kernel: [ 161.251104] RBP: ffffb9cbc11dbbe8 R08: 0000000000000000 R09: 0000000000000000 Apr 29 17:15:01 cluster1 kernel: [ 161.251106] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Apr 29 17:15:01 cluster1 kernel: [ 161.251111] R13: ffff941d893e9980 R14: 0000000000000000 R15: ffff941d80ad7a80 Apr 29 17:15:01 cluster1 kernel: [ 161.251113] FS: 00007c7dcbdb8a00(0000) GS:ffff941ef7c80000(0000) knlGS:0000000000000000 Apr 29 17:15:01 cluster1 kernel: [ 161.251115] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 29 17:15:01 cluster1 kernel: [ 161.251117] CR2: 00005a30877ae488 CR3: 0000000105888003 CR4: 0000000000170ee0 Apr 29 17:15:01 cluster1 kernel: [ 161.251122] Call Trace: Apr 29 17:15:01 cluster1 kernel: [ 161.251128] Apr 29 17:15:01 cluster1 kernel: [ 161.251133] ? show_regs+0x6d/0x80 Apr 29 17:15:01 cluster1 kernel: [ 161.251145] ? __warn+0x89/0x160 Apr 29 17:15:01 cluster1 kernel: [ 161.251152] ? rcu_note_context_switch+0x2a7/0x2f0 Apr 29 17:15:01 cluster1 kernel: [ 161.251155] ? report_bug+0x17e/0x1b0 Apr 29 17:15:01 cluster1 kernel: [ 161.251172] ? handle_bug+0x46/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251187] ? exc_invalid_op+0x18/0x80 Apr 29 17:15:01 cluster1 kernel: [ 161.251190] ? asm_exc_invalid_op+0x1b/0x20 Apr 29 17:15:01 cluster1 kernel: [ 161.251202] ? rcu_note_context_switch+0x2a7/0x2f0 Apr 29 17:15:01 cluster1 kernel: [ 161.251205] ? rcu_note_context_switch+0x2a7/0x2f0 Apr 29 17:15:01 cluster1 kernel: [ 161.251208] __schedule+0xcc/0x750 Apr 29 17:15:01 cluster1 kernel: [ 161.251218] schedule+0x63/0x110 Apr 29 17:15:01 cluster1 kernel: [ 161.251222] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:15:01 cluster1 kernel: [ 161.251238] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:15:01 cluster1 kernel: [ 161.251245] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:15:01 cluster1 kernel: [ 161.251248] ep_poll+0x33f/0x390 Apr 29 17:15:01 cluster1 kernel: [ 161.251254] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:15:01 cluster1 kernel: [ 161.251257] do_epoll_wait+0xdb/0x100 Apr 29 17:15:01 cluster1 kernel: [ 161.251259] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:15:01 cluster1 kernel: [ 161.251265] do_syscall_64+0x5b/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251270] ? do_epoll_ctl+0x3cb/0x860 Apr 29 17:15:01 cluster1 kernel: [ 161.251273] ? __task_pid_nr_ns+0x6c/0xc0 Apr 29 17:15:01 cluster1 kernel: [ 161.251279] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:15:01 cluster1 kernel: [ 161.251284] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:15:01 cluster1 kernel: [ 161.251286] ? do_syscall_64+0x67/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251288] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:15:01 cluster1 kernel: [ 161.251300] ? do_syscall_64+0x67/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251304] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:15:01 cluster1 kernel: [ 161.251306] ? do_syscall_64+0x67/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251309] ? do_syscall_64+0x67/0x90 Apr 29 17:15:01 cluster1 kernel: [ 161.251313] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:15:01 cluster1 kernel: [ 161.251316] RIP: 0033:0x7c7dcf325dea Apr 29 17:15:01 cluster1 kernel: [ 161.251333] Code: 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 e8 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 5e c3 0f 1f 44 00 00 48 83 ec 28 89 54 24 18 Apr 29 17:15:01 cluster1 kernel: [ 161.251335] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:15:01 cluster1 kernel: [ 161.251338] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:15:01 cluster1 kernel: [ 161.251340] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:15:01 cluster1 kernel: [ 161.251341] RBP: 00005a3087794590 R08: 00000000000f423f R09: 00007ffdde5e0357 Apr 29 17:15:01 cluster1 kernel: [ 161.251343] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:15:01 cluster1 kernel: [ 161.251345] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:15:01 cluster1 kernel: [ 161.251348] Apr 29 17:15:01 cluster1 kernel: [ 161.251349] ---[ end trace 0000000000000000 ]--- Apr 29 17:15:07 cluster1 dnsmasq[1085]: read /etc/hosts - 9 names Apr 29 17:15:07 cluster1 dnsmasq-dhcp[1085]: read /var/lib/lxd/networks/lxdfan0/dnsmasq.hosts/test.eth0 Apr 29 17:15:07 cluster1 kernel: [ 167.920360] audit: type=1400 audit(1714428907.904:38): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_archive-var-lib-lxd-storage-pools-local-containers-test" pid=1702 comm="apparmor_parser" Apr 29 17:15:08 cluster1 kernel: [ 168.059414] audit: type=1400 audit(1714428908.044:39): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd_archive-var-lib-lxd-storage-pools-local-containers-test" pid=1707 comm="apparmor_parser" Apr 29 17:15:08 cluster1 kernel: [ 168.085086] audit: type=1400 audit(1714428908.068:40): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_archive-var-lib-lxd-storage-pools-local-containers-test-rootfs" pid=1711 comm="apparmor_parser" Apr 29 17:16:01 cluster1 kernel: [ 221.258499] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:16:01 cluster1 kernel: [ 221.259086] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/21:b..l Apr 29 17:16:01 cluster1 kernel: [ 221.259721] rcu: (detected by 2, t=15002 jiffies, g=11101, q=5863 ncpus=8) Apr 29 17:16:01 cluster1 kernel: [ 221.260308] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:16:01 cluster1 kernel: [ 221.260312] Call Trace: Apr 29 17:16:01 cluster1 kernel: [ 221.260314] Apr 29 17:16:01 cluster1 kernel: [ 221.260318] __schedule+0x2cb/0x750 Apr 29 17:16:01 cluster1 kernel: [ 221.260326] schedule+0x63/0x110 Apr 29 17:16:01 cluster1 kernel: [ 221.260328] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:16:01 cluster1 kernel: [ 221.260332] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:16:01 cluster1 kernel: [ 221.260337] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:16:01 cluster1 kernel: [ 221.260339] ep_poll+0x33f/0x390 Apr 29 17:16:01 cluster1 kernel: [ 221.260343] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:16:01 cluster1 kernel: [ 221.260346] do_epoll_wait+0xdb/0x100 Apr 29 17:16:01 cluster1 kernel: [ 221.260349] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:16:01 cluster1 kernel: [ 221.260352] do_syscall_64+0x5b/0x90 Apr 29 17:16:01 cluster1 kernel: [ 221.260356] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:16:01 cluster1 kernel: [ 221.260358] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:16:01 cluster1 kernel: [ 221.260360] ? do_syscall_64+0x67/0x90 Apr 29 17:16:01 cluster1 kernel: [ 221.260363] ? __task_pid_nr_ns+0x6c/0xc0 Apr 29 17:16:01 cluster1 kernel: [ 221.260367] ? __task_pid_nr_ns+0x6c/0xc0 Apr 29 17:16:01 cluster1 kernel: [ 221.260369] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:16:01 cluster1 kernel: [ 221.260371] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:16:01 cluster1 kernel: [ 221.260372] ? do_syscall_64+0x67/0x90 Apr 29 17:16:01 cluster1 kernel: [ 221.260375] ? do_syscall_64+0x67/0x90 Apr 29 17:16:01 cluster1 kernel: [ 221.260377] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:16:01 cluster1 kernel: [ 221.260380] RIP: 0033:0x7c7dcf325dea Apr 29 17:16:01 cluster1 kernel: [ 221.260406] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:16:01 cluster1 kernel: [ 221.260408] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:16:01 cluster1 kernel: [ 221.260410] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:16:01 cluster1 kernel: [ 221.260411] RBP: 00005a3087794590 R08: 00000000000f423f R09: 00007ffdde5e0357 Apr 29 17:16:01 cluster1 kernel: [ 221.260412] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:16:01 cluster1 kernel: [ 221.260414] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:16:01 cluster1 kernel: [ 221.260417] Apr 29 17:17:01 cluster1 CRON[1728]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Apr 29 17:18:23 cluster1 kernel: [ 363.831460] INFO: task unsquashfs:1712 blocked for more than 120 seconds. Apr 29 17:18:23 cluster1 kernel: [ 363.832091] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:18:23 cluster1 kernel: [ 363.832822] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:18:23 cluster1 kernel: [ 363.833503] task:unsquashfs state:D stack:0 pid:1712 ppid:1009 flags:0x00000006 Apr 29 17:18:23 cluster1 kernel: [ 363.833508] Call Trace: Apr 29 17:18:23 cluster1 kernel: [ 363.833510] Apr 29 17:18:23 cluster1 kernel: [ 363.833514] __schedule+0x2cb/0x750 Apr 29 17:18:23 cluster1 kernel: [ 363.833521] ? crypto_shash_update+0x29/0x70 Apr 29 17:18:23 cluster1 kernel: [ 363.833537] schedule+0x63/0x110 Apr 29 17:18:23 cluster1 kernel: [ 363.833540] schedule_timeout+0x157/0x170 Apr 29 17:18:23 cluster1 kernel: [ 363.833546] wait_for_completion+0x88/0x150 Apr 29 17:18:23 cluster1 kernel: [ 363.833549] __wait_rcu_gp+0x150/0x160 Apr 29 17:18:23 cluster1 kernel: [ 363.833560] synchronize_rcu+0x12d/0x140 Apr 29 17:18:23 cluster1 kernel: [ 363.833563] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:18:23 cluster1 kernel: [ 363.833565] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:18:23 cluster1 kernel: [ 363.833567] expand_files+0x85/0x230 Apr 29 17:18:23 cluster1 kernel: [ 363.833575] ? __check_object_size+0x23/0x30 Apr 29 17:18:23 cluster1 kernel: [ 363.833582] alloc_fd+0x53/0x180 Apr 29 17:18:23 cluster1 kernel: [ 363.833585] get_unused_fd_flags+0x29/0x40 Apr 29 17:18:23 cluster1 kernel: [ 363.833587] do_sys_openat2+0x91/0xe0 Apr 29 17:18:23 cluster1 kernel: [ 363.833590] __x64_sys_openat+0x6c/0xa0 Apr 29 17:18:23 cluster1 kernel: [ 363.833592] do_syscall_64+0x5b/0x90 Apr 29 17:18:23 cluster1 kernel: [ 363.833595] ? do_syscall_64+0x67/0x90 Apr 29 17:18:23 cluster1 kernel: [ 363.833597] ? do_syscall_64+0x67/0x90 Apr 29 17:18:23 cluster1 kernel: [ 363.833600] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:18:23 cluster1 kernel: [ 363.833602] ? do_syscall_64+0x67/0x90 Apr 29 17:18:23 cluster1 kernel: [ 363.833604] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:18:23 cluster1 kernel: [ 363.833607] RIP: 0033:0x7f7e277145b4 Apr 29 17:18:23 cluster1 kernel: [ 363.833622] RSP: 002b:00007fff65d4f060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101 Apr 29 17:18:23 cluster1 kernel: [ 363.833624] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f7e277145b4 Apr 29 17:18:23 cluster1 kernel: [ 363.833626] RDX: 0000000000000241 RSI: 00005d073104c620 RDI: 00000000ffffff9c Apr 29 17:18:23 cluster1 kernel: [ 363.833627] RBP: 00005d073104c620 R08: 0000000000000000 R09: 0000000000000000 Apr 29 17:18:23 cluster1 kernel: [ 363.833628] R10: 00000000000001a4 R11: 0000000000000293 R12: 0000000000000241 Apr 29 17:18:23 cluster1 kernel: [ 363.833629] R13: 00005d073104c620 R14: 0000000000000000 R15: 00005d073104c1b0 Apr 29 17:18:23 cluster1 kernel: [ 363.833632] Apr 29 17:19:01 cluster1 kernel: [ 401.269452] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:19:01 cluster1 kernel: [ 401.270092] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/43:b..l Apr 29 17:19:01 cluster1 kernel: [ 401.270753] rcu: (detected by 6, t=60007 jiffies, g=11101, q=15744 ncpus=8) Apr 29 17:19:01 cluster1 kernel: [ 401.271408] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:19:01 cluster1 kernel: [ 401.271412] Call Trace: Apr 29 17:19:01 cluster1 kernel: [ 401.271415] Apr 29 17:19:01 cluster1 kernel: [ 401.271419] __schedule+0x2cb/0x750 Apr 29 17:19:01 cluster1 kernel: [ 401.271438] schedule+0x63/0x110 Apr 29 17:19:01 cluster1 kernel: [ 401.271441] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:19:01 cluster1 kernel: [ 401.271445] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:19:01 cluster1 kernel: [ 401.271450] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:19:01 cluster1 kernel: [ 401.271452] ep_poll+0x33f/0x390 Apr 29 17:19:01 cluster1 kernel: [ 401.271457] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:19:01 cluster1 kernel: [ 401.271460] do_epoll_wait+0xdb/0x100 Apr 29 17:19:01 cluster1 kernel: [ 401.271463] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:19:01 cluster1 kernel: [ 401.271466] do_syscall_64+0x5b/0x90 Apr 29 17:19:01 cluster1 kernel: [ 401.271470] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:19:01 cluster1 kernel: [ 401.271472] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:19:01 cluster1 kernel: [ 401.271475] ? do_syscall_64+0x67/0x90 Apr 29 17:19:01 cluster1 kernel: [ 401.271477] ? do_syscall_64+0x67/0x90 Apr 29 17:19:01 cluster1 kernel: [ 401.271480] ? do_syscall_64+0x67/0x90 Apr 29 17:19:01 cluster1 kernel: [ 401.271482] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:19:01 cluster1 kernel: [ 401.271488] RIP: 0033:0x7c7dcf325dea Apr 29 17:19:01 cluster1 kernel: [ 401.271505] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:19:01 cluster1 kernel: [ 401.271507] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:19:01 cluster1 kernel: [ 401.271508] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:19:01 cluster1 kernel: [ 401.271510] RBP: 00005a3087794590 R08: 00000000000f423f R09: 00007ffdde5e0357 Apr 29 17:19:01 cluster1 kernel: [ 401.271511] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:19:01 cluster1 kernel: [ 401.271512] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:19:01 cluster1 kernel: [ 401.271515] Apr 29 17:20:24 cluster1 kernel: [ 484.661591] INFO: task systemd:1 blocked for more than 120 seconds. Apr 29 17:20:24 cluster1 kernel: [ 484.662986] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:20:24 cluster1 kernel: [ 484.664658] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:20:24 cluster1 kernel: [ 484.666334] task:systemd state:D stack:0 pid:1 ppid:0 flags:0x00000002 Apr 29 17:20:24 cluster1 kernel: [ 484.666340] Call Trace: Apr 29 17:20:24 cluster1 kernel: [ 484.666344] Apr 29 17:20:24 cluster1 kernel: [ 484.666349] __schedule+0x2cb/0x750 Apr 29 17:20:24 cluster1 kernel: [ 484.666358] schedule+0x63/0x110 Apr 29 17:20:24 cluster1 kernel: [ 484.666362] schedule_timeout+0x157/0x170 Apr 29 17:20:24 cluster1 kernel: [ 484.666368] wait_for_completion+0x88/0x150 Apr 29 17:20:24 cluster1 kernel: [ 484.666374] __wait_rcu_gp+0x150/0x160 Apr 29 17:20:24 cluster1 kernel: [ 484.666381] synchronize_rcu+0x12d/0x140 Apr 29 17:20:24 cluster1 kernel: [ 484.666386] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:20:24 cluster1 kernel: [ 484.666390] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:20:24 cluster1 kernel: [ 484.666395] rcu_sync_enter+0x59/0x120 Apr 29 17:20:24 cluster1 kernel: [ 484.666400] ? _kstrtoull+0x43/0xa0 Apr 29 17:20:24 cluster1 kernel: [ 484.666420] percpu_down_write+0x2b/0x1d0 Apr 29 17:20:24 cluster1 kernel: [ 484.666425] cgroup_procs_write_start+0x116/0x190 Apr 29 17:20:24 cluster1 kernel: [ 484.666436] __cgroup_procs_write+0x61/0x190 Apr 29 17:20:24 cluster1 kernel: [ 484.666439] cgroup_procs_write+0x17/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.666442] cgroup_file_write+0x8d/0x1b0 Apr 29 17:20:24 cluster1 kernel: [ 484.666448] ? __check_object_size.part.0+0x72/0x150 Apr 29 17:20:24 cluster1 kernel: [ 484.666453] kernfs_fop_write_iter+0x133/0x210 Apr 29 17:20:24 cluster1 kernel: [ 484.666464] vfs_write+0x254/0x440 Apr 29 17:20:24 cluster1 kernel: [ 484.666467] ksys_write+0x73/0x100 Apr 29 17:20:24 cluster1 kernel: [ 484.666469] __x64_sys_write+0x19/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.666472] do_syscall_64+0x5b/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.666475] ? count_memcg_events.constprop.0+0x2a/0x50 Apr 29 17:20:24 cluster1 kernel: [ 484.666482] ? handle_mm_fault+0xad/0x360 Apr 29 17:20:24 cluster1 kernel: [ 484.666487] ? do_user_addr_fault+0x238/0x6b0 Apr 29 17:20:24 cluster1 kernel: [ 484.666492] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:20:24 cluster1 kernel: [ 484.666495] ? irqentry_exit_to_user_mode+0x17/0x20 Apr 29 17:20:24 cluster1 kernel: [ 484.666497] ? irqentry_exit+0x43/0x50 Apr 29 17:20:24 cluster1 kernel: [ 484.666499] ? exc_page_fault+0x94/0x1b0 Apr 29 17:20:24 cluster1 kernel: [ 484.666501] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:20:24 cluster1 kernel: [ 484.666503] RIP: 0033:0x72ac011148bf Apr 29 17:20:24 cluster1 kernel: [ 484.666519] RSP: 002b:00007ffc4aae7e50 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Apr 29 17:20:24 cluster1 kernel: [ 484.666521] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000072ac011148bf Apr 29 17:20:24 cluster1 kernel: [ 484.666525] RDX: 0000000000000005 RSI: 00007ffc4aae800a RDI: 0000000000000019 Apr 29 17:20:24 cluster1 kernel: [ 484.666527] RBP: 00007ffc4aae800a R08: 0000000000000000 R09: 00007ffc4aae7e90 Apr 29 17:20:24 cluster1 kernel: [ 484.666528] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000005 Apr 29 17:20:24 cluster1 kernel: [ 484.666529] R13: 00005bf4a456f520 R14: 000072ac01216a00 R15: 0000000000000005 Apr 29 17:20:24 cluster1 kernel: [ 484.666532] Apr 29 17:20:24 cluster1 kernel: [ 484.666554] INFO: task systemd-journal:177 blocked for more than 120 seconds. Apr 29 17:20:24 cluster1 kernel: [ 484.668143] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:20:24 cluster1 kernel: [ 484.669798] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:20:24 cluster1 kernel: [ 484.671375] task:systemd-journal state:D stack:0 pid:177 ppid:1 flags:0x00000002 Apr 29 17:20:24 cluster1 kernel: [ 484.671379] Call Trace: Apr 29 17:20:24 cluster1 kernel: [ 484.671381] Apr 29 17:20:24 cluster1 kernel: [ 484.671383] __schedule+0x2cb/0x750 Apr 29 17:20:24 cluster1 kernel: [ 484.671388] schedule+0x63/0x110 Apr 29 17:20:24 cluster1 kernel: [ 484.671390] schedule_preempt_disabled+0x15/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.671393] __mutex_lock.constprop.0+0x3f8/0x7a0 Apr 29 17:20:24 cluster1 kernel: [ 484.671401] __mutex_lock_slowpath+0x13/0x20 Apr 29 17:20:24 cluster1 kernel: [ 484.671403] mutex_lock+0x3c/0x50 Apr 29 17:20:24 cluster1 kernel: [ 484.671405] proc_cgroup_show+0x4c/0x440 Apr 29 17:20:24 cluster1 kernel: [ 484.671408] proc_single_show+0x56/0xe0 Apr 29 17:20:24 cluster1 kernel: [ 484.671414] seq_read_iter+0x135/0x4a0 Apr 29 17:20:24 cluster1 kernel: [ 484.671419] ? cp_new_stat+0x143/0x180 Apr 29 17:20:24 cluster1 kernel: [ 484.671422] seq_read+0xcd/0x110 Apr 29 17:20:24 cluster1 kernel: [ 484.671424] vfs_read+0xb4/0x360 Apr 29 17:20:24 cluster1 kernel: [ 484.671426] ? __seccomp_filter+0x37b/0x560 Apr 29 17:20:24 cluster1 kernel: [ 484.671435] ksys_read+0x73/0x100 Apr 29 17:20:24 cluster1 kernel: [ 484.671437] __x64_sys_read+0x19/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.671439] do_syscall_64+0x5b/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.671442] ? ksys_read+0x73/0x100 Apr 29 17:20:24 cluster1 kernel: [ 484.671444] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:20:24 cluster1 kernel: [ 484.671446] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:20:24 cluster1 kernel: [ 484.671448] ? do_syscall_64+0x67/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.671451] ? do_syscall_64+0x67/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.671453] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:20:24 cluster1 kernel: [ 484.671456] RIP: 0033:0x724c1bf1481c Apr 29 17:20:24 cluster1 kernel: [ 484.671467] RSP: 002b:00007ffcc37ca540 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 Apr 29 17:20:24 cluster1 kernel: [ 484.671469] RAX: ffffffffffffffda RBX: 0000638dbb175240 RCX: 0000724c1bf1481c Apr 29 17:20:24 cluster1 kernel: [ 484.671470] RDX: 0000000000000400 RSI: 0000638dbb15e620 RDI: 000000000000001a Apr 29 17:20:24 cluster1 kernel: [ 484.671472] RBP: 0000724c1c017600 R08: 0000000000000000 R09: 0000000000000001 Apr 29 17:20:24 cluster1 kernel: [ 484.671473] R10: 0000000000001000 R11: 0000000000000246 R12: 0000724c1c0336c8 Apr 29 17:20:24 cluster1 kernel: [ 484.671474] R13: 0000000000000d68 R14: 0000724c1c016a00 R15: 0000000000000d68 Apr 29 17:20:24 cluster1 kernel: [ 484.671476] Apr 29 17:20:24 cluster1 kernel: [ 484.671496] INFO: task unsquashfs:1712 blocked for more than 241 seconds. Apr 29 17:20:24 cluster1 kernel: [ 484.672178] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:20:24 cluster1 kernel: [ 484.672860] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:20:24 cluster1 kernel: [ 484.673617] task:unsquashfs state:D stack:0 pid:1712 ppid:1009 flags:0x00000006 Apr 29 17:20:24 cluster1 kernel: [ 484.673620] Call Trace: Apr 29 17:20:24 cluster1 kernel: [ 484.673622] Apr 29 17:20:24 cluster1 kernel: [ 484.673628] __schedule+0x2cb/0x750 Apr 29 17:20:24 cluster1 kernel: [ 484.673632] ? crypto_shash_update+0x29/0x70 Apr 29 17:20:24 cluster1 kernel: [ 484.673636] schedule+0x63/0x110 Apr 29 17:20:24 cluster1 kernel: [ 484.673639] schedule_timeout+0x157/0x170 Apr 29 17:20:24 cluster1 kernel: [ 484.673642] wait_for_completion+0x88/0x150 Apr 29 17:20:24 cluster1 kernel: [ 484.673645] __wait_rcu_gp+0x150/0x160 Apr 29 17:20:24 cluster1 kernel: [ 484.673648] synchronize_rcu+0x12d/0x140 Apr 29 17:20:24 cluster1 kernel: [ 484.673651] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:20:24 cluster1 kernel: [ 484.673653] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:20:24 cluster1 kernel: [ 484.673655] expand_files+0x85/0x230 Apr 29 17:20:24 cluster1 kernel: [ 484.673659] ? __check_object_size+0x23/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.673662] alloc_fd+0x53/0x180 Apr 29 17:20:24 cluster1 kernel: [ 484.673664] get_unused_fd_flags+0x29/0x40 Apr 29 17:20:24 cluster1 kernel: [ 484.673667] do_sys_openat2+0x91/0xe0 Apr 29 17:20:24 cluster1 kernel: [ 484.673670] __x64_sys_openat+0x6c/0xa0 Apr 29 17:20:24 cluster1 kernel: [ 484.673672] do_syscall_64+0x5b/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.673675] ? do_syscall_64+0x67/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.673677] ? do_syscall_64+0x67/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.673679] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:20:24 cluster1 kernel: [ 484.673681] ? do_syscall_64+0x67/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.673684] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:20:24 cluster1 kernel: [ 484.673686] RIP: 0033:0x7f7e277145b4 Apr 29 17:20:24 cluster1 kernel: [ 484.673693] RSP: 002b:00007fff65d4f060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101 Apr 29 17:20:24 cluster1 kernel: [ 484.673696] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f7e277145b4 Apr 29 17:20:24 cluster1 kernel: [ 484.673697] RDX: 0000000000000241 RSI: 00005d073104c620 RDI: 00000000ffffff9c Apr 29 17:20:24 cluster1 kernel: [ 484.673698] RBP: 00005d073104c620 R08: 0000000000000000 R09: 0000000000000000 Apr 29 17:20:24 cluster1 kernel: [ 484.673699] R10: 00000000000001a4 R11: 0000000000000293 R12: 0000000000000241 Apr 29 17:20:24 cluster1 kernel: [ 484.673700] R13: 00005d073104c620 R14: 0000000000000000 R15: 00005d073104c1b0 Apr 29 17:20:24 cluster1 kernel: [ 484.673703] Apr 29 17:20:24 cluster1 kernel: [ 484.673705] INFO: task (wnloader):1736 blocked for more than 120 seconds. Apr 29 17:20:24 cluster1 kernel: [ 484.674336] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:20:24 cluster1 kernel: [ 484.674996] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:20:24 cluster1 kernel: [ 484.675685] task:(wnloader) state:D stack:0 pid:1736 ppid:1 flags:0x00000002 Apr 29 17:20:24 cluster1 kernel: [ 484.675692] Call Trace: Apr 29 17:20:24 cluster1 kernel: [ 484.675693] Apr 29 17:20:24 cluster1 kernel: [ 484.675694] __schedule+0x2cb/0x750 Apr 29 17:20:24 cluster1 kernel: [ 484.675698] schedule+0x63/0x110 Apr 29 17:20:24 cluster1 kernel: [ 484.675700] schedule_preempt_disabled+0x15/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.675703] __mutex_lock.constprop.0+0x3f8/0x7a0 Apr 29 17:20:24 cluster1 kernel: [ 484.675714] ? _raw_spin_unlock+0xe/0x40 Apr 29 17:20:24 cluster1 kernel: [ 484.675718] __mutex_lock_slowpath+0x13/0x20 Apr 29 17:20:24 cluster1 kernel: [ 484.675721] mutex_lock+0x3c/0x50 Apr 29 17:20:24 cluster1 kernel: [ 484.675724] cgroup_kn_lock_live+0x4a/0xf0 Apr 29 17:20:24 cluster1 kernel: [ 484.675735] __cgroup_procs_write+0x42/0x190 Apr 29 17:20:24 cluster1 kernel: [ 484.675740] cgroup_procs_write+0x17/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.675745] cgroup_file_write+0x8d/0x1b0 Apr 29 17:20:24 cluster1 kernel: [ 484.675759] ? __check_object_size.part.0+0x72/0x150 Apr 29 17:20:24 cluster1 kernel: [ 484.675764] kernfs_fop_write_iter+0x133/0x210 Apr 29 17:20:24 cluster1 kernel: [ 484.675766] vfs_write+0x254/0x440 Apr 29 17:20:24 cluster1 kernel: [ 484.675769] ksys_write+0x73/0x100 Apr 29 17:20:24 cluster1 kernel: [ 484.675772] __x64_sys_write+0x19/0x30 Apr 29 17:20:24 cluster1 kernel: [ 484.675773] do_syscall_64+0x5b/0x90 Apr 29 17:20:24 cluster1 kernel: [ 484.675776] ? exc_page_fault+0x94/0x1b0 Apr 29 17:20:24 cluster1 kernel: [ 484.675778] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:20:24 cluster1 kernel: [ 484.675780] RIP: 0033:0x72ac011148bf Apr 29 17:20:24 cluster1 kernel: [ 484.675786] RSP: 002b:00007ffc4aae7ad0 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Apr 29 17:20:24 cluster1 kernel: [ 484.675788] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000072ac011148bf Apr 29 17:20:24 cluster1 kernel: [ 484.675798] RDX: 0000000000000005 RSI: 00007ffc4aae7c8a RDI: 0000000000000003 Apr 29 17:20:24 cluster1 kernel: [ 484.675799] RBP: 00007ffc4aae7c8a R08: 0000000000000000 R09: 00007ffc4aae7b10 Apr 29 17:20:24 cluster1 kernel: [ 484.675800] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000005 Apr 29 17:20:24 cluster1 kernel: [ 484.675801] R13: 00005bf4a456f520 R14: 000072ac01216a00 R15: 0000000000000005 Apr 29 17:20:24 cluster1 kernel: [ 484.675804] Apr 29 17:22:01 cluster1 kernel: [ 581.282742] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:22:01 cluster1 kernel: [ 581.283350] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/43:b..l Apr 29 17:22:01 cluster1 kernel: [ 581.284035] rcu: (detected by 5, t=105012 jiffies, g=11101, q=25580 ncpus=8) Apr 29 17:22:01 cluster1 kernel: [ 581.284655] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:22:01 cluster1 kernel: [ 581.284659] Call Trace: Apr 29 17:22:01 cluster1 kernel: [ 581.284662] Apr 29 17:22:01 cluster1 kernel: [ 581.284666] __schedule+0x2cb/0x750 Apr 29 17:22:01 cluster1 kernel: [ 581.284673] schedule+0x63/0x110 Apr 29 17:22:01 cluster1 kernel: [ 581.284676] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:22:01 cluster1 kernel: [ 581.284679] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:22:01 cluster1 kernel: [ 581.284685] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:22:01 cluster1 kernel: [ 581.284687] ep_poll+0x33f/0x390 Apr 29 17:22:01 cluster1 kernel: [ 581.284691] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:22:01 cluster1 kernel: [ 581.284694] do_epoll_wait+0xdb/0x100 Apr 29 17:22:01 cluster1 kernel: [ 581.284696] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:22:01 cluster1 kernel: [ 581.284700] do_syscall_64+0x5b/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284703] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:22:01 cluster1 kernel: [ 581.284706] ? do_syscall_64+0x67/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284708] ? do_syscall_64+0x67/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284710] ? do_syscall_64+0x67/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284712] ? do_syscall_64+0x67/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284715] ? do_syscall_64+0x67/0x90 Apr 29 17:22:01 cluster1 kernel: [ 581.284717] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:22:01 cluster1 kernel: [ 581.284720] RIP: 0033:0x7c7dcf325dea Apr 29 17:22:01 cluster1 kernel: [ 581.284735] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:22:01 cluster1 kernel: [ 581.284737] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:22:01 cluster1 kernel: [ 581.284738] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:22:01 cluster1 kernel: [ 581.284739] RBP: 00005a3087794590 R08: 00000000000f423f R09: 00007ffdde5e0357 Apr 29 17:22:01 cluster1 kernel: [ 581.284740] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:22:01 cluster1 kernel: [ 581.284741] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:22:01 cluster1 kernel: [ 581.284744] Apr 29 17:22:25 cluster1 kernel: [ 605.486270] INFO: task systemd:1 blocked for more than 241 seconds. Apr 29 17:22:25 cluster1 kernel: [ 605.486871] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:22:25 cluster1 kernel: [ 605.487537] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:22:25 cluster1 kernel: [ 605.488250] task:systemd state:D stack:0 pid:1 ppid:0 flags:0x00000002 Apr 29 17:22:25 cluster1 kernel: [ 605.488255] Call Trace: Apr 29 17:22:25 cluster1 kernel: [ 605.488257] Apr 29 17:22:25 cluster1 kernel: [ 605.488261] __schedule+0x2cb/0x750 Apr 29 17:22:25 cluster1 kernel: [ 605.488268] schedule+0x63/0x110 Apr 29 17:22:25 cluster1 kernel: [ 605.488271] schedule_timeout+0x157/0x170 Apr 29 17:22:25 cluster1 kernel: [ 605.488274] wait_for_completion+0x88/0x150 Apr 29 17:22:25 cluster1 kernel: [ 605.488277] __wait_rcu_gp+0x150/0x160 Apr 29 17:22:25 cluster1 kernel: [ 605.488282] synchronize_rcu+0x12d/0x140 Apr 29 17:22:25 cluster1 kernel: [ 605.488285] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:22:25 cluster1 kernel: [ 605.488287] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:22:25 cluster1 kernel: [ 605.488290] rcu_sync_enter+0x59/0x120 Apr 29 17:22:25 cluster1 kernel: [ 605.488292] ? _kstrtoull+0x43/0xa0 Apr 29 17:22:25 cluster1 kernel: [ 605.488296] percpu_down_write+0x2b/0x1d0 Apr 29 17:22:25 cluster1 kernel: [ 605.488299] cgroup_procs_write_start+0x116/0x190 Apr 29 17:22:25 cluster1 kernel: [ 605.488302] __cgroup_procs_write+0x61/0x190 Apr 29 17:22:25 cluster1 kernel: [ 605.488305] cgroup_procs_write+0x17/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.488308] cgroup_file_write+0x8d/0x1b0 Apr 29 17:22:25 cluster1 kernel: [ 605.488311] ? __check_object_size.part.0+0x72/0x150 Apr 29 17:22:25 cluster1 kernel: [ 605.488315] kernfs_fop_write_iter+0x133/0x210 Apr 29 17:22:25 cluster1 kernel: [ 605.488318] vfs_write+0x254/0x440 Apr 29 17:22:25 cluster1 kernel: [ 605.488322] ksys_write+0x73/0x100 Apr 29 17:22:25 cluster1 kernel: [ 605.488324] __x64_sys_write+0x19/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.488326] do_syscall_64+0x5b/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.488329] ? count_memcg_events.constprop.0+0x2a/0x50 Apr 29 17:22:25 cluster1 kernel: [ 605.488332] ? handle_mm_fault+0xad/0x360 Apr 29 17:22:25 cluster1 kernel: [ 605.488335] ? do_user_addr_fault+0x238/0x6b0 Apr 29 17:22:25 cluster1 kernel: [ 605.488338] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:22:25 cluster1 kernel: [ 605.488352] ? irqentry_exit_to_user_mode+0x17/0x20 Apr 29 17:22:25 cluster1 kernel: [ 605.488355] ? irqentry_exit+0x43/0x50 Apr 29 17:22:25 cluster1 kernel: [ 605.488356] ? exc_page_fault+0x94/0x1b0 Apr 29 17:22:25 cluster1 kernel: [ 605.488358] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:22:25 cluster1 kernel: [ 605.488361] RIP: 0033:0x72ac011148bf Apr 29 17:22:25 cluster1 kernel: [ 605.488377] RSP: 002b:00007ffc4aae7e50 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Apr 29 17:22:25 cluster1 kernel: [ 605.488379] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000072ac011148bf Apr 29 17:22:25 cluster1 kernel: [ 605.488381] RDX: 0000000000000005 RSI: 00007ffc4aae800a RDI: 0000000000000019 Apr 29 17:22:25 cluster1 kernel: [ 605.488382] RBP: 00007ffc4aae800a R08: 0000000000000000 R09: 00007ffc4aae7e90 Apr 29 17:22:25 cluster1 kernel: [ 605.488383] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000005 Apr 29 17:22:25 cluster1 kernel: [ 605.488384] R13: 00005bf4a456f520 R14: 000072ac01216a00 R15: 0000000000000005 Apr 29 17:22:25 cluster1 kernel: [ 605.488386] Apr 29 17:22:25 cluster1 kernel: [ 605.488398] INFO: task systemd-journal:177 blocked for more than 241 seconds. Apr 29 17:22:25 cluster1 kernel: [ 605.489010] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:22:25 cluster1 kernel: [ 605.489652] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:22:25 cluster1 kernel: [ 605.490455] task:systemd-journal state:D stack:0 pid:177 ppid:1 flags:0x00000002 Apr 29 17:22:25 cluster1 kernel: [ 605.490463] Call Trace: Apr 29 17:22:25 cluster1 kernel: [ 605.490465] Apr 29 17:22:25 cluster1 kernel: [ 605.490467] __schedule+0x2cb/0x750 Apr 29 17:22:25 cluster1 kernel: [ 605.490471] schedule+0x63/0x110 Apr 29 17:22:25 cluster1 kernel: [ 605.490473] schedule_preempt_disabled+0x15/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.490476] __mutex_lock.constprop.0+0x3f8/0x7a0 Apr 29 17:22:25 cluster1 kernel: [ 605.490479] __mutex_lock_slowpath+0x13/0x20 Apr 29 17:22:25 cluster1 kernel: [ 605.490480] mutex_lock+0x3c/0x50 Apr 29 17:22:25 cluster1 kernel: [ 605.490482] proc_cgroup_show+0x4c/0x440 Apr 29 17:22:25 cluster1 kernel: [ 605.490485] proc_single_show+0x56/0xe0 Apr 29 17:22:25 cluster1 kernel: [ 605.490488] seq_read_iter+0x135/0x4a0 Apr 29 17:22:25 cluster1 kernel: [ 605.490490] ? cp_new_stat+0x143/0x180 Apr 29 17:22:25 cluster1 kernel: [ 605.490494] seq_read+0xcd/0x110 Apr 29 17:22:25 cluster1 kernel: [ 605.490496] vfs_read+0xb4/0x360 Apr 29 17:22:25 cluster1 kernel: [ 605.490498] ? __seccomp_filter+0x37b/0x560 Apr 29 17:22:25 cluster1 kernel: [ 605.490503] ksys_read+0x73/0x100 Apr 29 17:22:25 cluster1 kernel: [ 605.490505] __x64_sys_read+0x19/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.490507] do_syscall_64+0x5b/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.490510] ? ksys_read+0x73/0x100 Apr 29 17:22:25 cluster1 kernel: [ 605.490511] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:22:25 cluster1 kernel: [ 605.490513] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:22:25 cluster1 kernel: [ 605.490515] ? do_syscall_64+0x67/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.490518] ? do_syscall_64+0x67/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.490520] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:22:25 cluster1 kernel: [ 605.490523] RIP: 0033:0x724c1bf1481c Apr 29 17:22:25 cluster1 kernel: [ 605.490532] RSP: 002b:00007ffcc37ca540 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 Apr 29 17:22:25 cluster1 kernel: [ 605.490534] RAX: ffffffffffffffda RBX: 0000638dbb175240 RCX: 0000724c1bf1481c Apr 29 17:22:25 cluster1 kernel: [ 605.490535] RDX: 0000000000000400 RSI: 0000638dbb15e620 RDI: 000000000000001a Apr 29 17:22:25 cluster1 kernel: [ 605.490536] RBP: 0000724c1c017600 R08: 0000000000000000 R09: 0000000000000001 Apr 29 17:22:25 cluster1 kernel: [ 605.490537] R10: 0000000000001000 R11: 0000000000000246 R12: 0000724c1c0336c8 Apr 29 17:22:25 cluster1 kernel: [ 605.490538] R13: 0000000000000d68 R14: 0000724c1c016a00 R15: 0000000000000d68 Apr 29 17:22:25 cluster1 kernel: [ 605.490550] Apr 29 17:22:25 cluster1 kernel: [ 605.490570] INFO: task unsquashfs:1712 blocked for more than 362 seconds. Apr 29 17:22:25 cluster1 kernel: [ 605.491165] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:22:25 cluster1 kernel: [ 605.491859] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:22:25 cluster1 kernel: [ 605.492522] task:unsquashfs state:D stack:0 pid:1712 ppid:1009 flags:0x00000006 Apr 29 17:22:25 cluster1 kernel: [ 605.492525] Call Trace: Apr 29 17:22:25 cluster1 kernel: [ 605.492526] Apr 29 17:22:25 cluster1 kernel: [ 605.492528] __schedule+0x2cb/0x750 Apr 29 17:22:25 cluster1 kernel: [ 605.492532] ? crypto_shash_update+0x29/0x70 Apr 29 17:22:25 cluster1 kernel: [ 605.492536] schedule+0x63/0x110 Apr 29 17:22:25 cluster1 kernel: [ 605.492538] schedule_timeout+0x157/0x170 Apr 29 17:22:25 cluster1 kernel: [ 605.492541] wait_for_completion+0x88/0x150 Apr 29 17:22:25 cluster1 kernel: [ 605.492544] __wait_rcu_gp+0x150/0x160 Apr 29 17:22:25 cluster1 kernel: [ 605.492547] synchronize_rcu+0x12d/0x140 Apr 29 17:22:25 cluster1 kernel: [ 605.492551] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:22:25 cluster1 kernel: [ 605.492557] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:22:25 cluster1 kernel: [ 605.492560] expand_files+0x85/0x230 Apr 29 17:22:25 cluster1 kernel: [ 605.492563] ? __check_object_size+0x23/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.492567] alloc_fd+0x53/0x180 Apr 29 17:22:25 cluster1 kernel: [ 605.492569] get_unused_fd_flags+0x29/0x40 Apr 29 17:22:25 cluster1 kernel: [ 605.492571] do_sys_openat2+0x91/0xe0 Apr 29 17:22:25 cluster1 kernel: [ 605.492574] __x64_sys_openat+0x6c/0xa0 Apr 29 17:22:25 cluster1 kernel: [ 605.492577] do_syscall_64+0x5b/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.492580] ? do_syscall_64+0x67/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.492582] ? do_syscall_64+0x67/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.492584] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:22:25 cluster1 kernel: [ 605.492586] ? do_syscall_64+0x67/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.492588] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:22:25 cluster1 kernel: [ 605.492590] RIP: 0033:0x7f7e277145b4 Apr 29 17:22:25 cluster1 kernel: [ 605.492598] RSP: 002b:00007fff65d4f060 EFLAGS: 00000293 ORIG_RAX: 0000000000000101 Apr 29 17:22:25 cluster1 kernel: [ 605.492602] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f7e277145b4 Apr 29 17:22:25 cluster1 kernel: [ 605.492604] RDX: 0000000000000241 RSI: 00005d073104c620 RDI: 00000000ffffff9c Apr 29 17:22:25 cluster1 kernel: [ 605.492607] RBP: 00005d073104c620 R08: 0000000000000000 R09: 0000000000000000 Apr 29 17:22:25 cluster1 kernel: [ 605.492608] R10: 00000000000001a4 R11: 0000000000000293 R12: 0000000000000241 Apr 29 17:22:25 cluster1 kernel: [ 605.492609] R13: 00005d073104c620 R14: 0000000000000000 R15: 00005d073104c1b0 Apr 29 17:22:25 cluster1 kernel: [ 605.492612] Apr 29 17:22:25 cluster1 kernel: [ 605.492614] INFO: task (wnloader):1736 blocked for more than 241 seconds. Apr 29 17:22:25 cluster1 kernel: [ 605.493193] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:22:25 cluster1 kernel: [ 605.493835] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:22:25 cluster1 kernel: [ 605.494619] task:(wnloader) state:D stack:0 pid:1736 ppid:1 flags:0x00000002 Apr 29 17:22:25 cluster1 kernel: [ 605.494622] Call Trace: Apr 29 17:22:25 cluster1 kernel: [ 605.494623] Apr 29 17:22:25 cluster1 kernel: [ 605.494625] __schedule+0x2cb/0x750 Apr 29 17:22:25 cluster1 kernel: [ 605.494629] schedule+0x63/0x110 Apr 29 17:22:25 cluster1 kernel: [ 605.494631] schedule_preempt_disabled+0x15/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.494633] __mutex_lock.constprop.0+0x3f8/0x7a0 Apr 29 17:22:25 cluster1 kernel: [ 605.494635] ? _raw_spin_unlock+0xe/0x40 Apr 29 17:22:25 cluster1 kernel: [ 605.494639] __mutex_lock_slowpath+0x13/0x20 Apr 29 17:22:25 cluster1 kernel: [ 605.494640] mutex_lock+0x3c/0x50 Apr 29 17:22:25 cluster1 kernel: [ 605.494642] cgroup_kn_lock_live+0x4a/0xf0 Apr 29 17:22:25 cluster1 kernel: [ 605.494645] __cgroup_procs_write+0x42/0x190 Apr 29 17:22:25 cluster1 kernel: [ 605.494648] cgroup_procs_write+0x17/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.494651] cgroup_file_write+0x8d/0x1b0 Apr 29 17:22:25 cluster1 kernel: [ 605.494654] ? __check_object_size.part.0+0x72/0x150 Apr 29 17:22:25 cluster1 kernel: [ 605.494657] kernfs_fop_write_iter+0x133/0x210 Apr 29 17:22:25 cluster1 kernel: [ 605.494660] vfs_write+0x254/0x440 Apr 29 17:22:25 cluster1 kernel: [ 605.494663] ksys_write+0x73/0x100 Apr 29 17:22:25 cluster1 kernel: [ 605.494665] __x64_sys_write+0x19/0x30 Apr 29 17:22:25 cluster1 kernel: [ 605.494667] do_syscall_64+0x5b/0x90 Apr 29 17:22:25 cluster1 kernel: [ 605.494670] ? exc_page_fault+0x94/0x1b0 Apr 29 17:22:25 cluster1 kernel: [ 605.494672] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:22:25 cluster1 kernel: [ 605.494673] RIP: 0033:0x72ac011148bf Apr 29 17:22:25 cluster1 kernel: [ 605.494693] RSP: 002b:00007ffc4aae7ad0 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Apr 29 17:22:25 cluster1 kernel: [ 605.494695] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000072ac011148bf Apr 29 17:22:25 cluster1 kernel: [ 605.494696] RDX: 0000000000000005 RSI: 00007ffc4aae7c8a RDI: 0000000000000003 Apr 29 17:22:25 cluster1 kernel: [ 605.494697] RBP: 00007ffc4aae7c8a R08: 0000000000000000 R09: 00007ffc4aae7b10 Apr 29 17:22:25 cluster1 kernel: [ 605.494698] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000005 Apr 29 17:22:25 cluster1 kernel: [ 605.494699] R13: 00005bf4a456f520 R14: 000072ac01216a00 R15: 0000000000000005 Apr 29 17:22:25 cluster1 kernel: [ 605.494702] Apr 29 17:24:26 cluster1 kernel: [ 726.319115] INFO: task systemd:1 blocked for more than 362 seconds. Apr 29 17:24:26 cluster1 kernel: [ 726.319712] Tainted: P W O 6.5.0-28-generic #29~22.04.1-Ubuntu Apr 29 17:24:26 cluster1 kernel: [ 726.320489] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 29 17:24:26 cluster1 kernel: [ 726.321171] task:systemd state:D stack:0 pid:1 ppid:0 flags:0x00000002 Apr 29 17:24:26 cluster1 kernel: [ 726.321175] Call Trace: Apr 29 17:24:26 cluster1 kernel: [ 726.321177] Apr 29 17:24:26 cluster1 kernel: [ 726.321181] __schedule+0x2cb/0x750 Apr 29 17:24:26 cluster1 kernel: [ 726.321189] schedule+0x63/0x110 Apr 29 17:24:26 cluster1 kernel: [ 726.321191] schedule_timeout+0x157/0x170 Apr 29 17:24:26 cluster1 kernel: [ 726.321195] wait_for_completion+0x88/0x150 Apr 29 17:24:26 cluster1 kernel: [ 726.321198] __wait_rcu_gp+0x150/0x160 Apr 29 17:24:26 cluster1 kernel: [ 726.321202] synchronize_rcu+0x12d/0x140 Apr 29 17:24:26 cluster1 kernel: [ 726.321206] ? __pfx_call_rcu_hurry+0x10/0x10 Apr 29 17:24:26 cluster1 kernel: [ 726.321208] ? __pfx_wakeme_after_rcu+0x10/0x10 Apr 29 17:24:26 cluster1 kernel: [ 726.321210] rcu_sync_enter+0x59/0x120 Apr 29 17:24:26 cluster1 kernel: [ 726.321213] ? _kstrtoull+0x43/0xa0 Apr 29 17:24:26 cluster1 kernel: [ 726.321217] percpu_down_write+0x2b/0x1d0 Apr 29 17:24:26 cluster1 kernel: [ 726.321219] cgroup_procs_write_start+0x116/0x190 Apr 29 17:24:26 cluster1 kernel: [ 726.321223] __cgroup_procs_write+0x61/0x190 Apr 29 17:24:26 cluster1 kernel: [ 726.321226] cgroup_procs_write+0x17/0x30 Apr 29 17:24:26 cluster1 kernel: [ 726.321229] cgroup_file_write+0x8d/0x1b0 Apr 29 17:24:26 cluster1 kernel: [ 726.321232] ? __check_object_size.part.0+0x72/0x150 Apr 29 17:24:26 cluster1 kernel: [ 726.321236] kernfs_fop_write_iter+0x133/0x210 Apr 29 17:24:26 cluster1 kernel: [ 726.321240] vfs_write+0x254/0x440 Apr 29 17:24:26 cluster1 kernel: [ 726.321243] ksys_write+0x73/0x100 Apr 29 17:24:26 cluster1 kernel: [ 726.321245] __x64_sys_write+0x19/0x30 Apr 29 17:24:26 cluster1 kernel: [ 726.321247] do_syscall_64+0x5b/0x90 Apr 29 17:24:26 cluster1 kernel: [ 726.321251] ? count_memcg_events.constprop.0+0x2a/0x50 Apr 29 17:24:26 cluster1 kernel: [ 726.321254] ? handle_mm_fault+0xad/0x360 Apr 29 17:24:26 cluster1 kernel: [ 726.321257] ? do_user_addr_fault+0x238/0x6b0 Apr 29 17:24:26 cluster1 kernel: [ 726.321259] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:24:26 cluster1 kernel: [ 726.321261] ? irqentry_exit_to_user_mode+0x17/0x20 Apr 29 17:24:26 cluster1 kernel: [ 726.321264] ? irqentry_exit+0x43/0x50 Apr 29 17:24:26 cluster1 kernel: [ 726.321265] ? exc_page_fault+0x94/0x1b0 Apr 29 17:24:26 cluster1 kernel: [ 726.321267] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:24:26 cluster1 kernel: [ 726.321269] RIP: 0033:0x72ac011148bf Apr 29 17:24:26 cluster1 kernel: [ 726.321286] RSP: 002b:00007ffc4aae7e50 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Apr 29 17:24:26 cluster1 kernel: [ 726.321296] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000072ac011148bf Apr 29 17:24:26 cluster1 kernel: [ 726.321297] RDX: 0000000000000005 RSI: 00007ffc4aae800a RDI: 0000000000000019 Apr 29 17:24:26 cluster1 kernel: [ 726.321301] RBP: 00007ffc4aae800a R08: 0000000000000000 R09: 00007ffc4aae7e90 Apr 29 17:24:26 cluster1 kernel: [ 726.321302] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000005 Apr 29 17:24:26 cluster1 kernel: [ 726.321303] R13: 00005bf4a456f520 R14: 000072ac01216a00 R15: 0000000000000005 Apr 29 17:24:26 cluster1 kernel: [ 726.321306] Apr 29 17:24:26 cluster1 kernel: [ 726.321307] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings Apr 29 17:25:01 cluster1 kernel: [ 761.297925] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:25:01 cluster1 kernel: [ 761.298531] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/45:b..l Apr 29 17:25:01 cluster1 kernel: [ 761.299212] rcu: (detected by 6, t=150017 jiffies, g=11101, q=137901 ncpus=8) Apr 29 17:25:01 cluster1 kernel: [ 761.299909] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:25:01 cluster1 kernel: [ 761.299913] Call Trace: Apr 29 17:25:01 cluster1 kernel: [ 761.299915] Apr 29 17:25:01 cluster1 kernel: [ 761.299919] __schedule+0x2cb/0x750 Apr 29 17:25:01 cluster1 kernel: [ 761.299926] schedule+0x63/0x110 Apr 29 17:25:01 cluster1 kernel: [ 761.299929] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:25:01 cluster1 kernel: [ 761.299932] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:25:01 cluster1 kernel: [ 761.299937] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:25:01 cluster1 kernel: [ 761.299940] ep_poll+0x33f/0x390 Apr 29 17:25:01 cluster1 kernel: [ 761.299944] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:25:01 cluster1 kernel: [ 761.299947] do_epoll_wait+0xdb/0x100 Apr 29 17:25:01 cluster1 kernel: [ 761.299949] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:25:01 cluster1 kernel: [ 761.299953] do_syscall_64+0x5b/0x90 Apr 29 17:25:01 cluster1 kernel: [ 761.299956] ? do_syscall_64+0x67/0x90 Apr 29 17:25:01 cluster1 kernel: [ 761.299959] ? do_syscall_64+0x67/0x90 Apr 29 17:25:01 cluster1 kernel: [ 761.299961] ? do_syscall_64+0x67/0x90 Apr 29 17:25:01 cluster1 kernel: [ 761.299963] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:25:01 cluster1 kernel: [ 761.299966] RIP: 0033:0x7c7dcf325dea Apr 29 17:25:01 cluster1 kernel: [ 761.299981] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:25:01 cluster1 kernel: [ 761.299984] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:25:01 cluster1 kernel: [ 761.299985] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:25:01 cluster1 kernel: [ 761.299987] RBP: 00005a3087794590 R08: 00000000000f423f R09: 00007ffdde5e0357 Apr 29 17:25:01 cluster1 kernel: [ 761.299988] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:25:01 cluster1 kernel: [ 761.299989] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:25:01 cluster1 kernel: [ 761.299992] Apr 29 17:28:01 cluster1 kernel: [ 941.311327] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:28:01 cluster1 kernel: [ 941.311918] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/47:b..l Apr 29 17:28:01 cluster1 kernel: [ 941.312525] rcu: (detected by 3, t=195022 jiffies, g=11101, q=148576 ncpus=8) Apr 29 17:28:01 cluster1 kernel: [ 941.313211] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:28:01 cluster1 kernel: [ 941.313216] Call Trace: Apr 29 17:28:01 cluster1 kernel: [ 941.313218] Apr 29 17:28:01 cluster1 kernel: [ 941.313222] __schedule+0x2cb/0x750 Apr 29 17:28:01 cluster1 kernel: [ 941.313229] schedule+0x63/0x110 Apr 29 17:28:01 cluster1 kernel: [ 941.313232] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:28:01 cluster1 kernel: [ 941.313235] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:28:01 cluster1 kernel: [ 941.313240] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:28:01 cluster1 kernel: [ 941.313242] ep_poll+0x33f/0x390 Apr 29 17:28:01 cluster1 kernel: [ 941.313246] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:28:01 cluster1 kernel: [ 941.313249] do_epoll_wait+0xdb/0x100 Apr 29 17:28:01 cluster1 kernel: [ 941.313263] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:28:01 cluster1 kernel: [ 941.313266] do_syscall_64+0x5b/0x90 Apr 29 17:28:01 cluster1 kernel: [ 941.313270] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:28:01 cluster1 kernel: [ 941.313272] ? do_syscall_64+0x67/0x90 Apr 29 17:28:01 cluster1 kernel: [ 941.313275] ? __x64_sys_epoll_ctl+0x6e/0xb0 Apr 29 17:28:01 cluster1 kernel: [ 941.313278] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:28:01 cluster1 kernel: [ 941.313280] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:28:01 cluster1 kernel: [ 941.313282] ? do_syscall_64+0x67/0x90 Apr 29 17:28:01 cluster1 kernel: [ 941.313284] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:28:01 cluster1 kernel: [ 941.313286] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:28:01 cluster1 kernel: [ 941.313287] ? do_syscall_64+0x67/0x90 Apr 29 17:28:01 cluster1 kernel: [ 941.313290] ? do_syscall_64+0x67/0x90 Apr 29 17:28:01 cluster1 kernel: [ 941.313292] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:28:01 cluster1 kernel: [ 941.313295] RIP: 0033:0x7c7dcf325dea Apr 29 17:28:01 cluster1 kernel: [ 941.313311] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:28:01 cluster1 kernel: [ 941.313313] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:28:01 cluster1 kernel: [ 941.313314] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:28:01 cluster1 kernel: [ 941.313316] RBP: 00005a3087794590 R08: 00000000000f423d R09: 00007ffdde5e0357 Apr 29 17:28:01 cluster1 kernel: [ 941.313317] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:28:01 cluster1 kernel: [ 941.313318] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:28:01 cluster1 kernel: [ 941.313321] Apr 29 17:31:01 cluster1 kernel: [ 1121.322852] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: Apr 29 17:31:01 cluster1 kernel: [ 1121.323511] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7): P510/47:b..l Apr 29 17:31:01 cluster1 kernel: [ 1121.324525] rcu: (detected by 1, t=240027 jiffies, g=11101, q=158056 ncpus=8) Apr 29 17:31:01 cluster1 kernel: [ 1121.325628] task:nmbd state:S stack:0 pid:510 ppid:1 flags:0x00000002 Apr 29 17:31:01 cluster1 kernel: [ 1121.325633] Call Trace: Apr 29 17:31:01 cluster1 kernel: [ 1121.325635] Apr 29 17:31:01 cluster1 kernel: [ 1121.325639] __schedule+0x2cb/0x750 Apr 29 17:31:01 cluster1 kernel: [ 1121.325647] schedule+0x63/0x110 Apr 29 17:31:01 cluster1 kernel: [ 1121.325649] schedule_hrtimeout_range_clock+0xbc/0x130 Apr 29 17:31:01 cluster1 kernel: [ 1121.325653] ? __pfx_hrtimer_wakeup+0x10/0x10 Apr 29 17:31:01 cluster1 kernel: [ 1121.325657] schedule_hrtimeout_range+0x13/0x30 Apr 29 17:31:01 cluster1 kernel: [ 1121.325660] ep_poll+0x33f/0x390 Apr 29 17:31:01 cluster1 kernel: [ 1121.325664] ? __pfx_ep_autoremove_wake_function+0x10/0x10 Apr 29 17:31:01 cluster1 kernel: [ 1121.325666] do_epoll_wait+0xdb/0x100 Apr 29 17:31:01 cluster1 kernel: [ 1121.325669] __x64_sys_epoll_wait+0x6f/0x110 Apr 29 17:31:01 cluster1 kernel: [ 1121.325672] do_syscall_64+0x5b/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325676] ? __task_pid_nr_ns+0x6c/0xc0 Apr 29 17:31:01 cluster1 kernel: [ 1121.325679] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:31:01 cluster1 kernel: [ 1121.325681] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:31:01 cluster1 kernel: [ 1121.325683] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325686] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:31:01 cluster1 kernel: [ 1121.325688] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325690] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325692] ? __task_pid_nr_ns+0x6c/0xc0 Apr 29 17:31:01 cluster1 kernel: [ 1121.325695] ? exit_to_user_mode_prepare+0x30/0xb0 Apr 29 17:31:01 cluster1 kernel: [ 1121.325703] ? syscall_exit_to_user_mode+0x37/0x60 Apr 29 17:31:01 cluster1 kernel: [ 1121.325705] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325710] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325712] ? do_syscall_64+0x67/0x90 Apr 29 17:31:01 cluster1 kernel: [ 1121.325714] entry_SYSCALL_64_after_hwframe+0x6e/0xd8 Apr 29 17:31:01 cluster1 kernel: [ 1121.325717] RIP: 0033:0x7c7dcf325dea Apr 29 17:31:01 cluster1 kernel: [ 1121.325734] RSP: 002b:00007ffdde5e0278 EFLAGS: 00000246 ORIG_RAX: 00000000000000e8 Apr 29 17:31:01 cluster1 kernel: [ 1121.325736] RAX: ffffffffffffffda RBX: 00005a30877a2ea0 RCX: 00007c7dcf325dea Apr 29 17:31:01 cluster1 kernel: [ 1121.325738] RDX: 0000000000000001 RSI: 00007ffdde5e02ac RDI: 0000000000000005 Apr 29 17:31:01 cluster1 kernel: [ 1121.325739] RBP: 00005a3087794590 R08: 00000000000f423e R09: 00007ffdde5e0357 Apr 29 17:31:01 cluster1 kernel: [ 1121.325740] R10: 00000000000003e8 R11: 0000000000000246 R12: 00005a30877a2f30 Apr 29 17:31:01 cluster1 kernel: [ 1121.325741] R13: 00000000000003e8 R14: 0000000000000090 R15: 000000000000000a Apr 29 17:31:01 cluster1 kernel: [ 1121.325744]