BUG: soft lockup - CPU#4 stuck for 22s! [rm:30630]

[Micah Gersten]

There were similar messages for CPU #2. This was a few minutes after starting the -29.46 kernel

ii linux-image-3.2.0-29-generic 3.2.0-29.46

Aug 13 01:59:51 voyager kernel: [ 1308.229091] BUG: soft lockup - CPU#4 stuck for 22s! [rm:30630]
Aug 13 01:59:51 voyager kernel: [ 1308.229100] Modules linked in: overlayfs xt_multiport ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
 nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm bnep rfcomm bluetooth parport_pc ppdev binfmt_misc dm_crypt snd_hda_codec_hdmi snd_hda_code
c_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq arc4 snd_timer uvcvideo videodev joydev snd_seq_device snd iwlwifi v4l2_compat_ioctl32 soundcore mac8021
1 cfg80211 mei(C) snd_page_alloc psmouse serio_raw mac_hid dm_multipath jmb38x_ms memstick sbs sbshc coretemp lp parport nbd dm_raid45 xor dm_mirror dm_region_hash dm_log usbhid hid nouveau ttm firewire_ohci drm_
kms_helper firewire_core jme crc_itu_t drm sdhci_pci sdhci i2c_algo_bit mxm_wmi wmi video
Aug 13 01:59:51 voyager kernel: [ 1308.229247] CPU 4
Aug 13 01:59:51 voyager kernel: [ 1308.229250] Modules linked in: overlayfs xt_multiport ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state
 nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm bnep rfcomm bluetooth parport_pc ppdev binfmt_misc dm_crypt snd_hda_codec_hdmi snd_hda_code
c_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq arc4 snd_timer uvcvideo videodev joydev snd_seq_device snd iwlwifi v4l2_compat_ioctl32 soundcore mac8021
1 cfg80211 mei(C) snd_page_alloc psmouse serio_raw mac_hid dm_multipath jmb38x_ms memstick sbs sbshc coretemp lp parport nbd dm_raid45 xor dm_mirror dm_region_hash dm_log usbhid hid nouveau ttm firewire_ohci drm_
kms_helper firewire_core jme crc_itu_t drm sdhci_pci sdhci i2c_algo_bit mxm_wmi wmi video
Aug 13 01:59:51 voyager kernel: [ 1308.229376]
Aug 13 01:59:51 voyager kernel: [ 1308.229381] Pid: 30630, comm: rm Tainted: G D C 3.2.0-29-generic #46-Ubuntu System76, Inc. Gazelle Professional/Gazelle Professional
Aug 13 01:59:51 voyager kernel: [ 1308.229394] RIP: 0010:[<ffffffff8103dc62>] [<ffffffff8103dc62>] __ticket_spin_lock+0x22/0x30
Aug 13 01:59:51 voyager kernel: [ 1308.229408] RSP: 0018:ffff88073ff8bdd8 EFLAGS: 00000297
Aug 13 01:59:51 voyager kernel: [ 1308.229413] RAX: 0000000000007b10 RBX: ffffffff8125e686 RCX: 000000000000a9cc
Aug 13 01:59:51 voyager kernel: [ 1308.229418] RDX: 0000000000007b11 RSI: 000000000001a740 RDI: ffffffff81c04480
Aug 13 01:59:51 voyager kernel: [ 1308.229423] RBP: ffff88073ff8bdd8 R08: ffffffff8125e686 R09: 1010000000000000
Aug 13 01:59:51 voyager kernel: [ 1308.229429] R10: 0000000000000000 R11: ffffffff81827250 R12: 0000000000600f10
Aug 13 01:59:51 voyager kernel: [ 1308.229434] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8807e617e000
Aug 13 01:59:51 voyager kernel: [ 1308.229441] FS: 00007fb8fd02c700(0000) GS:ffff88081f500000(0000) knlGS:0000000000000000
Aug 13 01:59:51 voyager kernel: [ 1308.229447] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 13 01:59:51 voyager kernel: [ 1308.229452] CR2: 00007fb8fcaf6f60 CR3: 0000000666038000 CR4: 00000000000406e0
Aug 13 01:59:51 voyager kernel: [ 1308.229457] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug 13 01:59:51 voyager kernel: [ 1308.229462] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug 13 01:59:51 voyager kernel: [ 1308.229468] Process rm (pid: 30630, threadinfo ffff88073ff8a000, task ffff8807c963dc00)
Aug 13 01:59:51 voyager kernel: [ 1308.229473] Stack:
Aug 13 01:59:51 voyager kernel: [ 1308.229476] ffff88073ff8bde8 ffffffff816598de ffff88073ff8be08 ffffffff81190b9c
Aug 13 01:59:51 voyager kernel: [ 1308.229487] ffffffff818275c0 ffff88065a318790 ffff88073ff8be38 ffffffff811923b7
Aug 13 01:59:51 voyager kernel: [ 1308.229498] ffff8807eb039800 ffff88065a318790 ffff88065a318810 ffffffff818275c0
Aug 13 01:59:51 voyager kernel: [ 1308.229508] Call Trace:
Aug 13 01:59:51 voyager kernel: [ 1308.229517] [<ffffffff816598de>] _raw_spin_lock+0xe/0x20
Aug 13 01:59:51 voyager kernel: [ 1308.229528] [<ffffffff81190b9c>] __remove_inode_hash+0x2c/0x90
Aug 13 01:59:51 voyager kernel: [ 1308.229536] [<ffffffff811923b7>] evict+0xe7/0x1c0
Aug 13 01:59:51 voyager kernel: [ 1308.229543] [<ffffffff81192578>] iput_final+0xe8/0x210
Aug 13 01:59:51 voyager kernel: [ 1308.229549] [<ffffffff811926de>] iput+0x3e/0x50
Aug 13 01:59:51 voyager kernel: [ 1308.229557] [<ffffffff81187123>] do_unlinkat+0x153/0x1d0
Aug 13 01:59:51 voyager kernel: [ 1308.229567] [<ffffffff8117ca4a>] ? sys_newfstatat+0x2a/0x40
Aug 13 01:59:51 voyager kernel: [ 1308.229576] [<ffffffff81187cc2>] sys_unlinkat+0x22/0x40
Aug 13 01:59:51 voyager kernel: [ 1308.229583] [<ffffffff81661ec2>] system_call_fastpath+0x16/0x1b
Aug 13 01:59:51 voyager kernel: [ 1308.229588] Code: 90 90 90 90 90 90 90 90 90 55 b8 00 00 01 00 48 89 e5 f0 0f c1 07 89 c2 c1 ea 10 66 39 c2 74 13 66 0f 1f 84 00 00 00 00 00 f3 90 <0f> b7 07 66 39 d0 75 f6 5d c3 0f 1f 40 00 8b 17 55 31 c0 48 89
Aug 13 01:59:51 voyager kernel: [ 1308.229671] Call Trace:
Aug 13 01:59:51 voyager kernel: [ 1308.229676] [<ffffffff816598de>] _raw_spin_lock+0xe/0x20
Aug 13 01:59:51 voyager kernel: [ 1308.229685] [<ffffffff81190b9c>] __remove_inode_hash+0x2c/0x90
Aug 13 01:59:51 voyager kernel: [ 1308.229692] [<ffffffff811923b7>] evict+0xe7/0x1c0
Aug 13 01:59:51 voyager kernel: [ 1308.229698] [<ffffffff81192578>] iput_final+0xe8/0x210
Aug 13 01:59:51 voyager kernel: [ 1308.229705] [<ffffffff811926de>] iput+0x3e/0x50
Aug 13 01:59:51 voyager kernel: [ 1308.229712] [<ffffffff81187123>] do_unlinkat+0x153/0x1d0
Aug 13 01:59:51 voyager kernel: [ 1308.229721] [<ffffffff8117ca4a>] ? sys_newfstatat+0x2a/0x40
Aug 13 01:59:51 voyager kernel: [ 1308.229729] [<ffffffff81187cc2>] sys_unlinkat+0x22/0x40
Aug 13 01:59:51 voyager kernel: [ 1308.229736] [<ffffffff81661ec2>] system_call_fastpath+0x16/0x1b

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1036063

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Micah Gersten (micahg)]

I can't run with the new kernel, I'm happy to provide any specific logs needed, but would prefer not to attach everything.

[Luis Henriques (henrix)]

Hi Micah, we would need at least the full kernel log (dmesg). The information you provided from dmesg indicates that this was not the initial OOPS/BUG (the 'D' flag in the "Tainted: G D C 3.2.0-29-generic..." line means exactly that).

[Luis Henriques (henrix)]

Also: is this reproducible? What are the steps to reproduce? What sort of filesystem(s) are you using?

[penalvch (penalvch)]

Micah Gersten, this bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? If so, could you please test for this with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ .

If it remains an issue, could you please run the following command in the development release from a Terminal (Applications->Accessories->Terminal), as it will automatically gather and attach updated debug information to this report:

apport-collect -p linux <replace-with-bug-number>

Also, could you please test the latest upstream kernel available (not the daily folder) following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this bug is fixed in the mainline kernel, please add the following tags:
kernel-fixed-upstream
kernel-fixed-upstream-VERSION-NUMBER

where VERSION-NUMBER is the version number of the kernel you tested. For example:
kernel-fixed-upstream-v3.13-rc1

This can be done by clicking on the yellow circle with a black pencil icon next to the word Tags located at the bottom of the bug description. As well, please remove the tag:
needs-upstream-testing

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-VERSION-NUMBER

As well, please remove the tag:
needs-upstream-testing

Once testing of the upstream kernel is complete, please mark this bug's Status as Confirmed. Please let us know your results. Thank you for your understanding.

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]