kernel crashed system freeze

[forall]

May 10 01:23:38 zeus kernel: [227848.986476] BUG: soft lockup - CPU#0 stuck for 11s! [sshd:10198]
May 10 01:23:38 zeus kernel: [227848.986536]
May 10 01:23:38 zeus kernel: [227848.986538] Pid: 10198, comm: sshd Not tainted (2.6.24-16-xen #2)
May 10 01:23:38 zeus kernel: [227848.986539] EIP: 0061:[ipv6:_spin_lock+0xa/0x10] EFLAGS: 00200286 CPU: 0
May 10 01:23:38 zeus kernel: [227848.986545] EIP is at _spin_lock+0xa/0x10
May 10 01:23:38 zeus kernel: [227848.986546] EAX: c1e8140c EBX: 00000000 ECX: c1e81400 EDX: ea3a0880
May 10 01:23:38 zeus kernel: [227848.986548] ESI: 56da0067 EDI: 00000000 EBP: c0477158 ESP: c4813de8
May 10 01:23:38 zeus kernel: [227848.986549] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
May 10 01:23:38 zeus kernel: [227848.986553] CR0: 80050033 CR2: b7f10300 CR3: 2a2c2000 CR4: 00000660
May 10 01:23:38 zeus kernel: [227848.986556] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
May 10 01:23:38 zeus kernel: [227848.986558] DR6: ffff0ff0 DR7: 00000400
May 10 01:23:38 zeus kernel: [227848.986559] [__do_fault+0x29c/0x680] __do_fault+0x29c/0x680
May 10 01:23:38 zeus kernel: [227848.986576] [handle_mm_fault+0x327/0x1330] handle_mm_fault+0x327/0x1330
May 10 01:23:38 zeus kernel: [227848.986581] [__udp_lib_get_port+0xe0/0x210] __udp_lib_get_port+0xe0/0x210
May 10 01:23:38 zeus kernel: [227848.986586] [ipv6:_spin_lock_bh+0x8/0x20] _spin_lock_bh+0x8/0x20
May 10 01:23:38 zeus kernel: [227848.986588] [ipv6:release_sock+0x12/0xa0] release_sock+0x12/0xa0
May 10 01:23:38 zeus kernel: [227848.986592] [usbcore:copy_to_user+0x30/0x540] copy_to_user+0x30/0x60
May 10 01:23:38 zeus kernel: [227848.986596] [move_addr_to_user+0x7e/0x90] move_addr_to_user+0x7e/0x90
May 10 01:23:38 zeus kernel: [227848.986601] [sys_getsockname+0xc6/0xe0] sys_getsockname+0xc6/0xe0
May 10 01:23:38 zeus kernel: [227848.986605] [x_tables:kmem_cache_alloc+0xd5/0x6a0] kmem_cache_alloc+0xd5/0x120
May 10 01:23:38 zeus kernel: [227848.986609] [do_page_fault+0x3bc/0xee0] do_page_fault+0x3bc/0xee0
May 10 01:23:38 zeus kernel: [227848.986614] [d_kill+0x38/0x50] d_kill+0x38/0x50
May 10 01:23:38 zeus kernel: [227848.986618] [ext3:dput+0x87/0x2d0] dput+0x87/0x100
May 10 01:23:38 zeus kernel: [227848.986620] [__fput+0x12a/0x1a0] __fput+0x12a/0x1a0
May 10 01:23:38 zeus kernel: [227848.986623] [fuse:mntput_no_expire+0x24/0x4220] mntput_no_expire+0x24/0xa0
May 10 01:23:38 zeus kernel: [227848.986627] [filp_close+0x49/0x80] filp_close+0x49/0x80
May 10 01:23:38 zeus kernel: [227848.986631] [do_page_fault+0x0/0xee0] do_page_fault+0x0/0xee0
May 10 01:23:38 zeus kernel: [227848.986633] [error_code+0x35/0x40] error_code+0x35/0x40
May 10 01:23:38 zeus kernel: [227848.986637] [vcc_ioctl+0xd0/0x2a0] vcc_ioctl+0xd0/0x2a0
May 10 01:23:38 zeus kernel: [227848.986642] =======================
May 10 01:23:51 zeus kernel: [227860.433684] BUG: soft lockup - CPU#0 stuck for 11s! [sshd:10198]

After something such as logs system responds only to ping, I can not log on through ssh, or physically from the console

System ubuntu 8.04, Linux zeus 2.6.24-16-xen,

[Jean-Baptiste Lallement (jibel)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately we can't fix it, because your description does not yet have enough information.

Please include the following additional information, if you have not already done so (pay attention to lspci's additional options), as required by the Ubuntu Kernel Team:
1. Please include the output of the command "uname -a" in your next response. It should be one, long line of text which includes the exact kernel version you're running, as well as the CPU architecture.
2. Please run the command "dmesg > dmesg.log" after a fresh boot and attach the resulting file "dmesg.log" to this bug report.
3. Please run the command "sudo lspci -vvnn > lspci-vvnn.log" and attach the resulting file "lspci-vvnn.log" to this bug report.

For your reference, the full description of procedures for kernel-related bug reports is available at https://wiki.ubuntu.com/KernelTeamBugPolicies Thanks in advance!

[forall (forall-stalowka)]

Sorry, now I attached others information

uname -a
Linux zeus 2.6.24-16-xen #2 SMP Sun Apr 20 05:49:43 JST 2008 i686 GNU/Linux

[forall (forall-stalowka)]

And dmesg.log

[Jean-Baptiste Lallement (jibel)]

Thanks.

When the crash occurs, can you try rebooting to the same kernel if it boots or a known working kernel and see if any backtrace was logged to /var/log/kern.log.0 .

Please attach this file if anything was captured. If this is not possible, supply a digital photo of the screen to capture the crash.

[forall (forall-stalowka)]

Welcome

I use only the kernel because I am running several virtual machines as a guest, but unfortunately I can not do this photo because the system does not respond to commands from the keyboard physically attached.

Albert

P.S In kern.log.0 I have the same bug that only points to another process

[Jean-Baptiste Lallement (jibel)]

If you're not using IPv6 can you add a line "blacklist ipv6" to /etc/modprobe.d/blacklist reboot and tell if it still occurs.

Thanks.

[Christian Bayle (christian-bayle)]

Looks like "blacklist ipv6" to /etc/modprobe.d/blacklist workaround the problem on a 2.6.26-17-xen kernel

[Christian Bayle (christian-bayle)]

Error, the bug is still there, it just occurs a bit less often

[Jean-Baptiste Lallement (jibel)]

Albert, does the soft lock still occurs when blacklisting ipv6 module ?

Christian, can you attach the file /var/log/kern.log containing the crash to this report.

Thanks

[Christian Bayle (christian-bayle)]

Thanks for the quick answer.

Sorry, but I was unable to reproduce the crash, as I reinstalled the test server. I suspect this to be linked in some kind with tap:aio. Maybe I forget to blacklist in domU or dom0, the first time. IO seems to be really slower than with file on an older kernel.

I still get this one https://bugs.launchpad.net/ubuntu/+source/linux/+bug/172632 but I'm unable to get any stack trace, except I could pg-up sometime and see the same message as pasted.

I finally used Etch with 2.6.18 kernel, used file instead of tap:aio (note that I had to patch hardy to generate proper xen config file (with tap:aio inside))
I backported Lenny xen 3.2 stuffs and it works like a charm without patching anything.
Looks like xen is far from stable on recent kernel.
Hardy domU works very well on this Debian/Etch + Xen 3.2 and all the annoying message about libc6 and tls deseapeared.

I was a bit deceived not to be able to run xen easily on a fresh Hardy using xen-tools.
Maybe a better idea to use an other virtualisation solution on Ubuntu or not to use tap:aio (disabled in Etch Kernel), any suggestion?

[Leann Ogasawara (leannogasawara)]

The Ubuntu Kernel Team is planning to move to the 2.6.27 kernel for the upcoming Intrepid Ibex 8.10 release. As a result, the kernel team would appreciate it if you could please test this newer 2.6.27 Ubuntu kernel. There are one of two ways you should be able to test:

1) If you are comfortable installing packages on your own, the linux-image-2.6.27-* package is currently available for you to install and test.

--or--

2) The upcoming Alpha5 for Intrepid Ibex 8.10 will contain this newer 2.6.27 Ubuntu kernel. Alpha5 is set to be released Thursday Sept 4. Please watch http://www.ubuntu.com/testing for Alpha5 to be announced. You should then be able to test via a LiveCD.

Please let us know immediately if this newer 2.6.27 kernel resolves the bug reported here or if the issue remains. More importantly, please open a new bug report for each new bug/regression introduced by the 2.6.27 kernel and tag the bug report with 'linux-2.6.27'. Also, please specifically note if the issue does or does not appear in the 2.6.26 kernel. Thanks again, we really appreicate your help and feedback.

[Ralph Janke (txwikinger)]

The Intrepid Ibex 8.10 Beta release was most recently announced - http://www.ubuntu.com/testing/intrepid/beta . It contains the 2.6.27 Ubuntu kernel. It would be great if you could test and verify if this is still an issue. The status is being set to Incomplete until we receive further feedback. Thanks.

[forall (forall-stalowka)]

Hi

Everybody who have problem with kernel-xen 2.6.24, to suggest installed kernel from debian lenny repository
http://packages.debian.org/lenny/xen-linux-system-2.6.26-1-xen-686

Today I installed this kernel from debian repository and until this time I don't have any problem, the system not crashed when I upgrade the installaed pakcages. I will see after long time of using and load system, if system will be not crashed.

[Launchpad Janitor (janitor)]

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.

[Stefan Bader (smb)]

This bug is being closed as it does not seem to be an issue any more, Please reopen if this is still an issue in the current Ubuntu release, Jaunty Jackalope 9.04 - http://www.ubuntu.com/getubuntu/download. If the issue remains in Jaunty, please test the latest upstream kernel build - https://wiki.ubuntu.com/KernelMainlineBuilds . To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.