server is stable under older kernel (3.2.0-29.46), but crashes under newer kernels

[Tang Dahong]

My PC is stable running Ubuntu Server 12.04.5 LTS using the old kernel 3.2.0-29.46. But it crashes whenever I tried to update the kernel to 3.2.0-65.99 or higher (including 14.04.2 LTS). So somewhere between 3.2.0-29.46 and 3.2.0-65.99, changes to the kernel caused my PC to crash (note: I have not tried kernels older than 3.2.0-65.99, so I don't know the exact version that started to cause the crash).

I think this is likely a hardware compatibility issue since I could not find any system logs regarding the crashes.

My PC is very old (Compaq Evo D510 SFF/07E4h, BIOS 68602 v2.14 08/15/2002), so I am not sure if this bug is worth troubleshooting.

In the meantime, I am happy with the performance of my PC running 3.2.0-29.46. But if anyone is interested in troubleshooting this bug, please let me know.
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version k3.13.0-32-generic.
ApportVersion: 2.0.1-0ubuntu17.6
Architecture: i386
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: ubuntu 3262 F.... pulseaudio
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info:
 Card hw:0 'I82801DBICH4'/'Intel 82801DB-ICH4 with AD1981A at irq 17'
   Mixer name : 'Analog Devices AD1981A'
   Components : 'AC97a:41445372'
   Controls : 26
   Simple ctrls : 17
CasperVersion: 1.315.1
DistroRelease: Ubuntu 12.04
IwConfig:
 lo no wireless extensions.

 eth0 no wireless extensions.
LiveMediaBuild: Ubuntu 12.04.5 LTS "Precise Pangolin" - Release i386 (20140807.1)
Lsusb:
 Bus 001 Device 002: ID 058f:6387 Alcor Micro Corp. Transcend JetFlash Flash Drive
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
 Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: Compaq Evo D510 SFF
MarkForUpload: True
Package: linux (not installed)
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: file=/cdrom/preseed/username.seed boot=casper cdrom-detect/try-usb=true noprompt floppy.allowed_drive_mask=0 ignore_uuid initrd=/casper/initrd.lz quiet splash -- BOOT_IMAGE=/casper/vmlinuz
ProcVersionSignature: Ubuntu 3.13.0-32.57~precise1-generic 3.13.11.4
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-32-generic N/A
 linux-backports-modules-3.13.0-32-generic N/A
 linux-firmware 1.79.16
RfKill:

Tags: precise running-unity
Uname: Linux 3.13.0-32-generic i686
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
dmi.bios.date: 08/15/2002
dmi.bios.vendor: Compaq
dmi.bios.version: 686O2 v2.14
dmi.board.name: 07E4h
dmi.board.vendor: Compaq
dmi.chassis.type: 15
dmi.chassis.vendor: Compaq
dmi.modalias: dmi:bvnCompaq:bvr686O2v2.14:bd08/15/2002:svnCompaq:pnEvoD510SFF:pvr:rvnCompaq:rn07E4h:rvr:cvnCompaq:ct15:cvr:
dmi.product.name: Evo D510 SFF
dmi.sys.vendor: Compaq

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1446724/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1446724

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tang Dahong (tang-dahong)]

I can't send the log files using "apport-collcet 1446724" as instructed by Brad Figg because the text based browser on my server cannot authenticate launchpad login. But it doesn't matter since the crashes are not logged by system logs anyway.

[Tang Dahong (tang-dahong)]

Does the desktop live environment have to be ubuntu? I have a live linux desktop (e.g., linux mint) on a USB stick, but it's not ubuntu. Would that work?

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

apport information

[Tang Dahong (tang-dahong)]

I updated BIOS per your suggestion, but my PC crashed under the newer kernels as usual.

Here is the output of your command:
68602 v2.21
05/28/2003

By the way, could you please let me know where I can find "the Bug Description" that you referred to? I searched everywhere but am unable to find it yet. Thanks.

[penalvch (penalvch)]

Tang Dahong, could you please test the latest upstream kernel available from the very top line at the top of the page (the release names are irrelevant for testing, and please do not test the daily folder) following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue.

If the test did not allow you to test to the issue (ex. you couldn't boot into the OS) please make a comment in your report about this, and continue to test the next most recent kernel version until you can test to the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this bug is fixed in the mainline kernel, please add the following tags by clicking on the yellow circle with a black pencil icon, next to the word Tags, located at the bottom of the report description:
kernel-fixed-upstream
kernel-fixed-upstream-X.Y-rcZ

Where XY and Z are numbers corresponding to the kernel version.

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-X.Y-rcZ

Once testing of the upstream kernel is complete, please mark this bug's Status as Confirmed. Please let us know your results.

Thank you for your understanding.

[Tang Dahong (tang-dahong)]

I was able to narrow down the exact kernel version that caused the crash: it is Ubuntu-3.2.0-32.51. So, something changed between 3.2.0-31 and 3.2.0-32 that caused the crash.

Unfortunately, I am unable to test the upstream mainline kernel of 3.2.0-32. I can install the mainline kernel's linux-image (but strangely not its linux-headers); however, for whatever reason, the mainline kernel does not show up in the boot menu.

[penalvch (penalvch)]

tang dahong, the latest mainline kernel to test is 4.1-rc6.

[Tang Dahong (tang-dahong)]

I can install the mainline linux-image, but not the headers. This is true for every mainline kernel I tried. Here is an example of the error message:

dpkg: dependency problems prevent configuration of linux-headers-4.1.0-040100rc6-generic:
 linux-headers-4.1.0-040100rc6-generic depends on linux-headers-4.1.0-040100rc6; however:
  Package linux-headers-4.1.0-040100rc6 is not installed.
dpkg: error processing linux-headers-4.1.0-040100rc6-generic (--install):
 dependency problems - leaving unconfigured

Also, none of the mainline linux-image installed showed up in the boot menu.

Do you have some ideas on how to fix the installation issue? If not, then can you please propose alternative ways of debugging? Thanks.

[penalvch (penalvch)]

Tang Dahong, there is a deemphasis of support for i386 upstream. Hence, could you please convert to amd64 variant via http://releases.ubuntu.com/ (preferably Trusty) and advise if this is still reproducible?

[Tang Dahong (tang-dahong)]

Well, my PC has a 32-bit CPU, it is not 64-bit capable. FYI - when I installed the 14.04.2 LTS kernel, the server crashed also. 3.2.0-31 is the latest kernel that is still stable on my PC.

[penalvch (penalvch)]

Tang Dahong, the next step is to fully commit bisect from kernel 3.2.0-29.46 to 3.2.0-65.99 in order to identify the last good kernel commit, followed immediately by the first bad one. This will allow for a more expedited analysis of the root cause of your issue. Could you please do this following https://wiki.ubuntu.com/Kernel/KernelBisection ?

Please note, finding adjacent kernel versions is not fully commit bisecting.

Thank you for your understanding.

Helpful bug reporting tips:
https://wiki.ubuntu.com/ReportingBugs

[Tang Dahong (tang-dahong)]

I already did these tests (that you call "bisect") between kernels 3.2.0-29 and 14.02.2. As far as I can tell, every kernel after 3.2.0-31.50 causes my PC to crash, and 3.2.0-32.51 is the first kernel release that causes this problem.

[penalvch (penalvch)]

Tang Dahong, you have started a bisect, but did not finish.

[Tang Dahong (tang-dahong)]

Well, I have tested the release-versions of the kernels. I don't think I am capable of bisecting the git commits from scratch. I can be a tester though. Can you, or someone, build the kernels from git commits so that I can test them?

[Tang Dahong (tang-dahong)]

My server which was running 3.2.0-31 just crashed after running non-stop for 2 weeks. I need to go back and re-test 3.2.0-30.

[Tang Dahong (tang-dahong)]

My server ran for more than 30 days without crashing using kernel 3.2.0-29-generic-pae, but it crashed using 3.2.0-30 after only a few days. So I think 3.2.0-29 is the only kernel that's stable on my PC.