Random machine lockups under load [jaunty] [karmic]

[Mark Grandi]

**NOTE** This is a new bug report, as Carey Underwood told me to start a new one. The old bug report in question was this one: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/355155

Ever since i installed jaunty, randomly my computer would just freeze, and even a alt+f1 or a alt+sysrq+reisub would not do anything, so i assume its a kernel problem. This just happened right now, and i had to restart my computer, so i ran 'ubuntu-bug linux' the second i restarted.

This has been reproduced under the jaunty live-cd, as well as a fresh install of karmic alpha-5

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04, 9.10a5
MachineType: System manufacturer P5Q-PRO
NonfreeKernelModules: nvidia
Package: linux-image-2.6.28-11-generic 2.6.28-11.40
ProcCmdLine: root=UUID=d9574ed9-a697-45be-b773-ad8a81d79b6b ro quiet splash vga=792
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.28-11.40-generic, 2.6.31-whatever-is-in-karmic-alpha-5
SourcePackage: linux

[Carey Underwood (cwillu)]

I have 3 questions for starters :)

1) On arch linux, what is the output of uname -a, and /proc/config (or /proc/config.gz)?

2) Can you download an Ubuntu 9.04 live cd, and see if you can reproduce the issue from the live session (without installing)? If so, I'll point you to a 9.10 live cd image to see if you can also reproduce it there.

3) When you still had it installed, did you try installing and booting a kernel from the ubuntu's mainline kernel ppa?

[Carey Underwood (cwillu)]

Also, do you have a second machine available? If so, we can set up a netconsole to allow logs to be captured up to the moment of the crash (which may otherwise be preventing critical log entries from making it to the disk).

[Mark Grandi (markgrandi)]

1: Ok, i have attached the requested files (here and the next attachment since you cant upload mulitple attachments per post)

2: I will start the ubuntu 9.04 live cd (64 bit) and leave that running while doing something intensive (like ripping a dvd or something) and see if it crashes there.

3: when i was running ubuntu, i was just using the plain included kernel, i did not try booting a kernel from the ubuntu mainline kernel ppa.

[Mark Grandi (markgrandi)]

oh, and yes i have another machine. But i have no idea how to set up a netconsole though.

[Carey Underwood (cwillu)]

Incidently, you can send email to <email address hidden>, and
attach multiple files to a single post that way

[Carey Underwood (cwillu)]

On the second machine, run
> netcat -l -u -p 6000

On the first machine, run
> modprobe netconsole netconsole=@/eth0,6000@192.168.1.101/
, substituting the second machine's ip address in for 192.168.1.102.

Then hit alt-sysrq-s, and you should see a statement about syncing the
disks on the second machine. If you see that message (or any other
messages), you can now try to trigger a crash, and attach any
information that shows up.

[Carey Underwood (cwillu)]

If you can't reproduce the issue under a live cd, I'm going to have
you reinstall ubuntu (to a different partition, or however you want to
do it), and then install a different kernel. I would expect that to
'solve' the issue right there; that will allow us to determine if the
issue is solved under karmic, and to eventually work out exactly which
changes caused the issue.

[Mark Grandi (markgrandi)]

Yeah, i just ran the live cd all over night while ripping some dvds and its still alive and kicking, so it might just only be with an installed partition. I shall install ubuntu and give the netconsole thing a shot, might take a while since i have school and work and what not =)

[Carey Underwood (cwillu)]

Well, the first thing is to get it to reproduce. Do an install, but
don't worry about the netconsole thing for the moment. If/when you
reproduce, then we'll try a different kernel, and if we can still
reproduce it, we'll worry about netconsole.

[Mark Grandi (markgrandi)]

well, good news, i got it to reproduce, i set it to rip some dvds overnight and it didn't even last 2 hours before it froze again.

[Carey Underwood (cwillu)]

Okay, that's good (in a somewhat perverse sense of the word :p)

Next step, can you download the Desktop CD (live cd) from
http://cdimage.ubuntu.com/releases/9.10/alpha-5/ and see if you can
reproduce it on that version?

Note that because it is an alpha build, there could potentially be
other crashers, and so if you _can_ reproduce it, try to give as many
details as possible about the nature of the hang (mouse cursor,
capslock light, responsiveness to alt-sysrq-reisub, blinking lights,
etc)

This will allow us to pin things down quite a bit. Given that we can
reproduce it from the 9.04 livecd, it's safe to say that it wasn't
just an odd corruption issue on your previous install.

[Mark Grandi (markgrandi)]

well, i managed to install karmic (yikes what a nightmare, the live cd is terrible! It kept auto mounting drives the second i would unmount them when i was trying to resize a partition to make room for karmic.......)

anyway, it seems that ffmpeg can't compile under this release of ubuntu (i read that GCC got updated? maybe that is causing it to throw a fit?) Anyway....since that was my go to case of doing something intensive, it might be a while for me to see if i can reproduce this bug, if it infact only happens when its under load....i will try something else tonight though.

[Mark Grandi (markgrandi)]

well, this is good news (i guess), i managed to crash it again (using a precompiled binary of handbrake, seeding a torrent, and running a little program called glxdragon, kinda like glxgears) all night. Same symptopms:

nothing responds, no mouse, no keyboard, anything
no hard drive activity
no flashing keyboard lights
alt+sysrq+reisub doesn't do anything.

[Carey Underwood (cwillu)]

Okay, that's actually bad news, but it's progress none-the-less :)

Can you install the kernel from
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/423379

Assuming you're on 32-bit, you'll need to download and install
"linux-headers..._i386.deb", "linux-headers..._all.deb" and
"linux-image..._i386.deb". After you've done that, reboot, run "uname
-a" to verify you're on this new kernel, and then try to duplicate the
hang again.

[Carey Underwood (cwillu)]

I'm sorry, that link was supposed to be http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31-rc8/

[Mark Grandi (markgrandi)]

you posted the wrong link , thats just a link to this bug report. did you mean the kernel ppa?

[Carey Underwood (cwillu)]

You saw the corrected link above, right?

[Mark Grandi (markgrandi)]

yeah i saw it

i installed the linux headers (64 bit) deb, the linux headers (all) deb, the linux source (all) deb, and the linux image (64 bit) deb. and when i try to boot, it doesn't even start up, it just hangs at this screen: (attached)

im not sure if its freaking out cause my menu.lst is not on the ubuntu file system, but rather on my arch one, so i can still boot arch and windows.

this is the entry of my menu.lst for the ubuntu 9.10 /w ppa kernel:

# Ubuntu 9.10 with kernel ppa
title Ubuntu 9.10 Alpha 5 PPA kernel
root (hd2,5)
kernel /boot/vmlinuz-2.6.31-020631rc8-generic root=/dev/disk/by-uuid/f052cc4d-da19-4168-950e-7ff39ac70fa5 ro quiet splash
initrd /boot/initrd.img-2.6.31-020631rc8-generic

[Carey Underwood (cwillu)]

Okay, lets go a different route.

Can I get you to blow away the 9.10 install and install 9.04 in its
place? After its installed, install the kernel images from
http://kernel.ubuntu.com
/~kernel-ppa/mainline/v2.6.31-rc8/

Also, I believe you originally reported this bug on a 32bit install?
If so, lets stick to 32bit installs for now, one less confounding
factor :)

[Carey Underwood (cwillu)]

@Polygon: still there? We're so close to having everything a developer would need to get started, I'd hate to see this get abandoned now.

[Mark Grandi (markgrandi)]

don't worry, i am still here =P

I have just been very busy this week with school and work, so if i don't try it tonight, it will defenitly happen over the weekend. sorry xD

[BlackJudas (dserban)]

Random thought.

I had installed mythbuntu and experienced the same hard lockups as Polygon has. When I enabled the native atheros (madwifi) driver, the lockups ceased. I see in his lspci that he too has an atheros card. It might be something.

[BlackJudas (dserban)]

Sorry, forgot to mention that it's a jaunty install. (ia32)

[Mark Grandi (markgrandi)]

I believe ubuntu uses the ath5k module by default, since ath_pci is restricted cause of its closed HAL interface or something

And, i am using arch linux with ath5k as well and there are no crashes, so i would think that is not the cause. But i shall try that new ubuntu kernel tommrow at the latest....

[Carey Underwood (cwillu)]

> I had installed mythbuntu and experienced the same hard lockups as
> Polygon has. When I enabled the native atheros (madwifi) driver, the
> lockups ceased. I see in his lspci that he too has an atheros card. It
> might be something.

BlackJudas: Thanks, I'll keep that in mind. The problem is that there
are literally hundreds of things that can cause _exactly_ the symptoms
that Polygon has.

The single most important thing right now is to narrow the problem
down between ubuntu's userspace, ubuntu's kernel patches ('sauce'),
and ubuntu's kernel configuration. Once we've got that nailed down, it
should be easy to get a developer to see the actual problem; without
doing that, we'll just be going on a goose chase. Yes, we may get
lucky, but judging from the bug this was forked from, I think we'll
stick with the slow-and-steady approach. :)

[Mark Grandi (markgrandi)]

I just installed 9.04 and installed the PPA kernel again, and it also hangs at the exact same spot the 9.10 install hung on with the ppa kernel...it just freezes at 'loading hardware drivers'.

[Mark Grandi (markgrandi)]

I managed to compile a custom kernel using the latest stable branch from kernel.org (which is 3.6.31), and i just used ubuntu's config file and skipped all the new features, so i shall do my stress tests and see if it hangs overnight.

[Mark Grandi (markgrandi)]

well, it managed to rip 16 episodes of a tv show without freezing, so i guess it doesn't have the bug...but then again, i dunno how accurate this is, since its custom compiled and i dunno if ubuntu has their own patches that they apply and stuff.

[Mark Grandi (markgrandi)]

Hi, any updates on this? id like to get this fixed before karmic ships if at all possible.

[Mark Grandi (markgrandi)]

you said we were 'so close to getting a developer involved', and yet no comments to this bug in 10 days. Sounds like its not getting fixed for karmic.

[Brad Figg (brad-figg)]

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in a supported series, please file a new bug.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.