Machine crashes with "kernel BUG ... raid10.c"

[Tapani Tarvainen]

Machine running Ubuntu 13.10 server, LSI SATA card, five disks + spares in RAID10 configuration, crashed hard.
Syslog (from remote syslog server, crash prevented it being written on local disk) reports

kernel BUG at /build/buildd/linux-3.11.0/drivers/md/raid10.c:351!

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1286871/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1286871

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tapani Tarvainen (ubuntu-tapani)]

Is there some way to run apport-collect without GUI?

[Tapani Tarvainen (ubuntu-tapani)]

I guess I'll have to file a separate bug report about apport-collect: it tries to use links or lynx, but doesn't work with them, because launchpad insists on referer field. :-(

[Tapani Tarvainen (ubuntu-tapani)]

So: I can't run apport-collect. If someone will point me to a document listing logs it would send I'll try to attach them manually.

[Tapani Tarvainen (ubuntu-tapani)]

I've got the apport-cli --save output; will it help anybody if I attach it here as is (26991 lines) or should I try to split it somehow?

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.13 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-rc5-trusty/

[Joseph Salisbury (jsalisbury)]

You can collect apport data from the terminal with:
apport-bug --save /tmp/report.1286871 linux

Then attach the report.1286871 file to the bug report.

[Tapani Tarvainen (ubuntu-tapani)]

Attached output from "apport-bug --save /tmp/report.1286871 linux".

[Tapani Tarvainen (ubuntu-tapani)]

I will try the upstream kernel later today. The problem is somewhat difficult to reproduce, however - it happens somewhat randomly, probably load-dependent and I've already migrated all critical stuff out of the machine and it hasn't crashed since. But I'll try to generate some test load to see if I can force it to happen.

[Tapani Tarvainen (ubuntu-tapani)]

No errors after 24 hours with upstream kernel. That doesn't prove much though as the machine had been running without problems for a week before crashing on Sunday (then it crashed twice within an hour). I'll try to put more load on it and keep it running for a few days, if nothing happens I'll try again with Ubuntu kernel.

[Tapani Tarvainen (ubuntu-tapani)]

The machine's been running with upstream kernel (3.14.0-031400rc5-generic) for almost six weeks now without problems. I haven't had time to do more tests with the old Ubuntu kernel, and probably won't until Trusty comes out, so I cannot tell if the kernel change made the difference or if the error was the result of some really rare combination of circumstances.