Soft lockup - cpu stuck - cifs_fattr_to_inode

Bug #1816838 reported by Félix C. Morency on 2019-02-20
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
linux-signed-hwe (Ubuntu)
Undecided
Unassigned

Bug Description

I'm getting a lot of CPU stuck error when writing on a CIFS (SMBv3) mount and need to reboot the machine(s) every time it happens. This error happens when I have a lot of small processes writing small files to the mount.

The CIFS mount options are

  sec=krb5,user=$USER,cruid=$UID,multiuser,nounix,mfsymlinks,resilienthandles,vers=3.0

I'm running

  Description: Ubuntu 16.04.5 LTS
  Release: 16.04

on kernel 4.15.0-45-generic #48~16.04.1-Ubuntu SMP Tue Jan 29 18:03:48 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Trace is:

[ 5668.105623] CPU: 20 PID: 12828 Comm: action-suid Tainted: G D I L 4.15.0-45-generic #48~16.04.1-Ubuntu
[ 5668.105623] Hardware name: HP ProLiant DL360 G7, BIOS P68 08/16/2015
[ 5668.105626] RIP: 0010:native_queued_spin_lock_slowpath+0x118/0x1a0
[ 5668.105626] RSP: 0018:ffffaab55b5039b8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff11
[ 5668.105628] RAX: 0000000000000000 RBX: ffff8f91fff5c358 RCX: ffff8f893bca3440
[ 5668.105629] RDX: ffff8f925f9a3440 RSI: 0000000000380101 RDI: ffff8f91fff5c3e0
[ 5668.105629] RBP: ffffaab55b5039b8 R08: 0000000000540000 R09: 0000000000000000
[ 5668.105631] R10: ffff8f92347fa000 R11: ffffffffc086d420 R12: ffff8f91fff5c3e0
[ 5668.105632] R13: ffff8f91fff5c2b0 R14: ffff8f91fff5c310 R15: ffffaab55b503a60
[ 5668.105633] FS: 00007ff936633700(0000) GS:ffff8f893bc80000(0000) knlGS:0000000000000000
[ 5668.105634] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5668.105635] CR2: 00007f25bcfe3750 CR3: 00000008f5760003 CR4: 00000000000206e0
[ 5668.105636] Call Trace:
[ 5668.105638] _raw_spin_lock+0x20/0x30
[ 5668.105651] cifs_fattr_to_inode+0x7d/0x5a0 [cifs]
[ 5668.105663] cifs_get_inode_info+0x22e/0xcb0 [cifs]
[ 5668.105675] cifs_revalidate_dentry_attr+0x1d5/0x250 [cifs]
[ 5668.105687] cifs_revalidate_dentry+0x13/0x30 [cifs]
[ 5668.105697] cifs_d_revalidate+0x27/0xb0 [cifs]
[ 5668.105699] lookup_fast+0x270/0x300
[ 5668.105701] ? __inode_permission+0x5b/0x160
[ 5668.105702] walk_component+0x49/0x360
[ 5668.105704] ? path_init+0x1d9/0x310
[ 5668.105705] path_lookupat+0x76/0x230
[ 5668.105707] filename_lookup+0xb8/0x1a0
[ 5668.105709] ? __check_object_size+0xfc/0x1a0
[ 5668.105711] ? strncpy_from_user+0x4d/0x170
[ 5668.105713] user_path_at_empty+0x36/0x40
[ 5668.105714] ? user_path_at_empty+0x36/0x40
[ 5668.105715] vfs_statx+0x77/0xe0
[ 5668.105717] SYSC_newlstat+0x3d/0x70
[ 5668.105719] SyS_newlstat+0xe/0x10
[ 5668.105720] do_syscall_64+0x73/0x130
[ 5668.105722] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[ 5668.105723] RIP: 0033:0x7ff936143c85
[ 5668.105724] RSP: 002b:00007ffd8bacace8 EFLAGS: 00000246 ORIG_RAX: 0000000000000006
[ 5668.105725] RAX: ffffffffffffffda RBX: 000055f5e961e880 RCX: 00007ff936143c85
[ 5668.105726] RDX: 00007ffd8bacad30 RSI: 00007ffd8bacad30 RDI: 000055f5e961d880
[ 5668.105727] RBP: 00007ffd8bacadf0 R08: 31383130325f7374 R09: 7473616d5f383132
[ 5668.105728] R10: 65343237365f7265 R11: 0000000000000246 R12: 000055f5e961d880
[ 5668.105728] R13: 000055f5e961d8cd R14: 000055f5e961d846 R15: 000055f5e961d86d
[ 5668.105729] Code: 12 48 c1 ea 0c 83 e8 01 83 e2 30 48 98 48 81 c2 40 34 02 00 48 03 14 c5 c0 c6 3a 82 48 89 0a 8b 41 08 85 c0 75 09 f3 90 8b 41 08 <85> c0 7
4 f7 4c 8b 09 4d 85 c9 74 08 41 0f 18 09 eb 02 f3 90 8b

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.15.0-45-generic 4.15.0-45.48~16.04.1
ProcVersionSignature: User Name 4.15.0-45.48~16.04.1-generic 4.15.18
Uname: Linux 4.15.0-45-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
Date: Wed Feb 20 13:56:05 2019
SourcePackage: linux-signed-hwe
UpgradeStatus: No upgrade log present (probably fresh install)

Félix C. Morency (colibry10) wrote :
Félix C. Morency (colibry10) wrote :
Download full text (5.2 KiB)

Still happening on mainline kernel 4.20.11

Feb 22 16:31:12 imk-node01 kernel: [ 1465.018937] watchdog: BUG: soft lockup - CPU#23 stuck for 22s! [cp:31557]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018938] Modules linked in: overlay cmac nls_utf8 cifs ccm fscache ipmi_ssif intel_powerclamp coretemp kvm_intel kvm irqbypass gpio_ich acpi_power_meter intel_cstate hpilo ipmi_si i7core_edac ipmi_devintf ipmi_msghandler lpc_ich serio_raw mac_hid ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear radeon i2c_algo_bit ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect crc32_pclmul sysimgblt ghash_clmulni_intel fb_sys_fops drm drm_panel_orientation_quirks aesni_intel cfbfillrect cfbimgblt cfbcopyarea aes_x86_64 crypto_simd fb psmouse cryptd fbdev glue_helper hpsa i2c_core bnx2 scsi_transport_sas
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018965] CPU: 23 PID: 31557 Comm: cp Tainted: G D I L 4.20.11-042011-generic #201902200535
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018966] Hardware name: HP ProLiant DL360 G7, BIOS P68 08/16/2015
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018969] RIP: 0010:native_queued_spin_lock_slowpath+0x60/0x1a0
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018970] Code: 6e f0 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 48 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 5d 66 89 07 c3 8b 37 81 fe 00 01
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018971] RSP: 0018:ffffab3356b439c8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018973] RAX: 0000000000540101 RBX: ffff9e173e960000 RCX: 000000000a12a9c6
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018974] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9e1a3334c840
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018974] RBP: ffffab3356b439c8 R08: ffff9e2334ca5800 R09: ffff9e2334ca5100
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018975] R10: ffffddca47d5e400 R11: ffffffffc0a25fc0 R12: ffff9e1a3334c840
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018976] R13: 000000000a12a9c6 R14: ffff9e173e960000 R15: ffff9e1a35ef5300
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018978] FS: 00007f6085222800(0000) GS:ffff9e233bac0000(0000) knlGS:0000000000000000
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018978] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018979] CR2: 00007f60846fb6e0 CR3: 00000011f159a004 CR4: 00000000000206e0
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018980] Call Trace:
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018983] _raw_spin_lock+0x1f/0x30
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018999] is_size_safe_to_change+0x3c/0xc0 [cifs]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019013] cifs_fattr_to_inode+0x15e/0x5a0 [cifs]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019027] cifs_get_inode_info+0x16d/0xb60 [cifs]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019041] cifs_open+0x3a3/0x9f0 [cifs]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019056] ? cifsF...

Read more...

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-signed-hwe (Ubuntu):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers