Kernel Oops - BUG: unable to handle kernel NULL pointer dereference at 0000000000000050; RIP: 0010:[<ffffffff81566940>] [<ffffffff81566940>] ip_xfrm_me_harder.part.8+0x40/0x110

[Kiall Mac Innes]

This is a fresh Oneiric Beta 2 install, all updates applied.

The Oops appears to be caused by KVM starting a VM, which was triggered by starting openstack-compute.

This seems fairly repeatable.. Let me know if you need any more info.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-server 3.0.0.12.14
ProcVersionSignature: Ubuntu 3.0.0-12.19-server 3.0.4
Uname: Linux 3.0.0-12-server x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 2011-10-07 18:31 seq
 crw-rw---- 1 root audio 116, 33 2011-10-07 18:31 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 1.23-0ubuntu2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Date: Fri Oct 7 18:32:26 2011
HibernationDevice: RESUME=UUID=f36741ab-b941-45e4-8a84-f88776d4b3fc
InstallationMedia: Ubuntu-Server 11.10 "Oneiric Ocelot" - Beta amd64 (20110921)
IwConfig: Error: [Errno 2] No such file or directory
MachineType: Dell Inc. PowerEdge 1950
PciMultimedia:

ProcEnviron:
 LANGUAGE=en_IE:en
 LANG=en_IE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.0.0-12-server root=/dev/mapper/hostname-root ro
RelatedPackageVersions:
 linux-restricted-modules-3.0.0-12-server N/A
 linux-backports-modules-3.0.0-12-server N/A
 linux-firmware 1.60
RfKill: Error: [Errno 2] No such file or directory
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/29/2008
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.3.1
dmi.board.name: 0NK937
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr2.3.1:bd04/29/2008:svnDellInc.:pnPowerEdge1950:pvr:rvnDellInc.:rn0NK937:rvrA00:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge 1950
dmi.sys.vendor: Dell Inc.

[Joseph Salisbury (jsalisbury)]

@Kiall

I noticed some of the Oops screen shot is cut off. Would it be possible to get a screen shot of the entire Oops message?

Also, do you happen to know if this has just started happening with Oneiric? Have you done any testing with earlier releases?

[Kiall Mac Innes (kiall)]

@Joseph: This was a fresh install replacing a older VMWare ESXi box for a trial of OpenStack, so I have no history with previous versions..

I have a better (I think) screesnshot in the office from last night, trying to screenshot a console via VNC via RDP on my laptop is a tad awkward ;) I'll attach it in a few hours.

[Brad Figg (brad-figg)]

Thank you for taking the time to file a bug report on this issue.

However, given the number of bugs that the Kernel Team receives during any development cycle it is impossible for us to review them all. Therefore, we occasionally resort to using automated bots to request further testing. This is such a request.

We have noted that there is a newer version of the development kernel than the one you last tested when this issue was found. Please test again with the newer kernel and indicate in the bug if this issue still exists or not.

If the bug still exists, change the bug status from Incomplete to Confirmed. If the bug no longer exists, change the bug status from Incomplete to Fix Released.

Thank you for your help, we really do appreciate it.

[Kiall Mac Innes (kiall)]

I've just tested with the latest kernel, still getting the same kernel oops...

I'm noticing this on a few servers .. 2x Dell 1950's and 1x HP DL165 G7, all three are experiencing kernel panics v.frequently..

[Kiall Mac Innes (kiall)]

Also - I can provide access to one of the servers for someone better equipped to debug :)

[Kiall Mac Innes (kiall)]

Just tested with the mainline 3.0.6-030006-generic kernel, still panicking...

http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.0.6-oneiric/linux-image-3.0.6-030006-generic_3.0.6-030006.201110050043_amd64.deb

and ...

Just tested with the mainline 3.1.0-0301rc9-generic kernel, this kernel after a quick test seems okay! Normally, I would have managed to crash the server by starting/stopping/starting/stopping this many VMs (Until now, 4x starts has always triggered the issue .. I'm on about 15 now.. )

http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.0.6-oneiric/linux-image-3.0.6-030006-generic_3.0.6-030006.201110050043_amd64.deb

[Kiall Mac Innes (kiall)]

Damn - 3.1.0-0301rc9-generic lasted *much* longer, but eventually ended with another Oops...

[Clint Byrum (clint-fewbar)]

I believe we saw this on an intel Emerald Ridge box w/ 40 cores recently as well with the latest Oneiric while at the OpenStack summit. This is actually one of Canonical's spare kernel build machines, and should be directly available for testing.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kiall Mac Innes (kiall)]

Okay - I've managed to get about 24 hours without a panic! (With the stock oneiric packages ie 3.0 kernel etc)

To "fix" this issue, I purged all OpenStack components, Wiped the OpenStack database and re-installed.

I say "fix" because obviously the Oops should never have occurred at all, regardless of my OpenStack mis-configuration.

[penalvch (penalvch)]

Kiall Mac Innes, thank you for reporting this bug and helping make Ubuntu better. This bug report is being closed due to your last comment regarding this being fixed by wiping the OpenStack database and re-installing. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.