Activity log for bug #929941

Date Who What changed Old value New value Message
2012-02-10 01:59:49 Matt Wilson bug added bug
2012-02-10 02:00:31 Matt Wilson bug added subscriber JC Lima
2012-02-10 02:01:22 Matt Wilson bug added subscriber Doug Mitchell
2012-02-10 02:04:24 Matt Wilson attachment added stack traces from instance running 2.6.32-341 (1/2) https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/929941/+attachment/2728701/+files/ubuntu-deadlock-2.6.32-341-1.txt
2012-02-10 02:05:04 Matt Wilson attachment added stack traces from instance running 2.6.32-341 (2/2) https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/929941/+attachment/2728702/+files/ubuntu-deadlock-2.6.32-341-2.txt
2012-02-10 02:06:03 Matt Wilson attachment added stack traces from instance running 2.6.32-342 (1/1) https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/929941/+attachment/2728703/+files/ubuntu-deadlock-2.6.32-342-1.txt
2012-02-10 02:07:06 Matt Wilson visibility public private
2012-02-10 05:41:13 Matt Wilson bug added subscriber Heroku
2012-02-10 05:41:26 Matt Wilson bug added subscriber Will Leinweber
2012-02-10 19:04:19 Noah Zoschke bug added subscriber Noah Zoschke
2012-02-10 19:33:42 Scott Moser visibility private public
2012-02-10 19:33:50 Scott Moser bug added subscriber Stefan Bader
2012-02-10 19:33:56 Scott Moser bug added subscriber Ben Howard
2012-02-10 19:38:03 Matt Wilson attachment added stack traces from instance running 2.6.32-316 https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/929941/+attachment/2730182/+files/i-804475e2.txt
2012-02-13 10:32:23 Stefan Bader linux-ec2 (Ubuntu): importance Undecided High
2012-02-13 10:32:23 Stefan Bader linux-ec2 (Ubuntu): status New Incomplete
2012-02-13 10:32:23 Stefan Bader linux-ec2 (Ubuntu): assignee Stefan Bader (stefan-bader-canonical)
2012-02-13 18:22:07 Matt Wilson summary Kernel deadlock in scheduler on m2.2xlarge EC2 instance Kernel deadlock in scheduler on m2.{2,4}xlarge EC2 instance
2012-02-13 18:24:14 Matt Wilson attachment added /proc/interrupts as an attachment https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/929941/+attachment/2736482/+files/proc-interrupts.txt
2012-03-12 20:35:58 Matt Wilson summary Kernel deadlock in scheduler on m2.{2,4}xlarge EC2 instance Kernel deadlock in scheduler on multiple EC2 instance types
2012-03-13 02:37:02 Ilan bug added subscriber Ilan
2012-05-09 22:45:33 Fabio Kung bug added subscriber Fabio Kung
2012-06-12 00:24:36 Cristian Gafton bug added subscriber Cristian Gafton
2012-06-12 21:56:47 Simon Kuhn bug added subscriber Simon Kuhn
2012-06-20 14:16:27 Todd bug added subscriber Todd
2012-06-25 09:22:21 Stefan Bader linux-ec2 (Ubuntu): status Incomplete In Progress
2012-06-25 09:29:10 Stefan Bader description After running for some indeterminate period of time, the 2.6.32-341-ec2 and 2.6.32-342-ec2 kernels stop responding when running on m2.2xlarge EC2 instances. No console output is emitted. Stack dumps gathered by examining CPU context information show that all VCPUs are stuck waiting on spinlocks. This could be a deadlock in the scheduling code. ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: linux-image-2.6.32-341-ec2 2.6.32-341.42 ProcVersionSignature: User Name 2.6.32-341.42-ec2 2.6.32.49+drm33.21 Uname: Linux 2.6.32-341-ec2 x86_64 Architecture: amd64 Date: Fri Feb 10 01:56:17 2012 Ec2AMI: ami-55dc0b3c Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1c Ec2InstanceType: m1.xlarge Ec2Kernel: aki-427d952b Ec2Ramdisk: unavailable ProcEnviron: LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: linux-ec2 SRU Justification: Impact: The version of Xen patches we currently use for the ec2 kernel have a serious flaw in the handling of nested spinlocks. This can result in a complete deadlock under certain workloads. Fix: The spinlock handling code has been substantially restructured in later versions of the patchset. The changes backport this but also enable the use of ticket-spinlocks (as we do now) when compiling with the compatibility level we use. Testcase: Not easy to reproduce. But feedback with the patchset applied (see comment #32) look good. -- After running for some indeterminate period of time, the 2.6.32-341-ec2 and 2.6.32-342-ec2 kernels stop responding when running on m2.2xlarge EC2 instances. No console output is emitted. Stack dumps gathered by examining CPU context information show that all VCPUs are stuck waiting on spinlocks. This could be a deadlock in the scheduling code. ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: linux-image-2.6.32-341-ec2 2.6.32-341.42 ProcVersionSignature: User Name 2.6.32-341.42-ec2 2.6.32.49+drm33.21 Uname: Linux 2.6.32-341-ec2 x86_64 Architecture: amd64 Date: Fri Feb 10 01:56:17 2012 Ec2AMI: ami-55dc0b3c Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1c Ec2InstanceType: m1.xlarge Ec2Kernel: aki-427d952b Ec2Ramdisk: unavailable ProcEnviron:  LANG=en_US.UTF-8  SHELL=/bin/bash SourcePackage: linux-ec2
2012-06-25 09:29:22 Stefan Bader bug added subscriber Ubuntu Stable Release Updates Team
2012-06-25 15:49:55 Stefan Bader linux-ec2 (Ubuntu): status In Progress Fix Committed
2012-06-27 23:44:53 Matt Zimmerman bug added subscriber Matt Zimmerman
2012-07-07 22:38:06 Launchpad Janitor branch linked lp:ubuntu/lucid-proposed/linux-ec2
2012-07-09 11:38:04 Luis Henriques tags amd64 apport-bug ec2-images lucid amd64 apport-bug ec2-images lucid verification-needed-lucid
2012-07-10 17:44:35 Stefan Bader tags amd64 apport-bug ec2-images lucid verification-needed-lucid amd64 apport-bug ec2-images lucid verification-done-lucid
2012-07-23 17:22:10 Launchpad Janitor linux-ec2 (Ubuntu): status Fix Committed Fix Released