| 2011-08-31 16:14:11 |
Scott Moser |
description |
We had two similar looking failures on EC2 testing of Beta Candidate for oneiric [1].
In [2], we did collect a 'terminated' log, that shows a the failure. In [3], the results are very similar to [2], but we did not get the terminated console log, so we can't really be sure.
useful console log at [4], which i'll attach also. The interesting section of log is:
[11963.827784] udevd[82]: starting version 173
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Gave up waiting for root device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Check root= (did the system wait for the right device?)
- Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-label/cloudimg-rootfs does not exist. Dropping to a shell!
BusyBox v1.18.4 (Ubuntu 1:1.18.4-2ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
...[repeated]...
udevd[91]: '/sbin/blkid -o udev -p /dev/xvda1' [102] terminated by signal 9 (Killed)
At that point, the test harness rebooted the system and it came back up properly.
[1] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/7/
[2] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/7/ARCH=amd64,REGION=eu-west-1,STORAGE=ebs,TEST=multi-part-ud,label=ubuntu-server-ec2-testing/artifact/
[3] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/ARCH=i386,REGION=eu-west-1,STORAGE=ebs,TEST=multi-part-ud,label=ubuntu-server-ec2-testing/lastBuild/artifact/
[4] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/7/ARCH=amd64,REGION=eu-west-1,STORAGE=ebs,TEST=multi-part-ud,label=ubuntu-server-ec2-testing/artifact/None/amd64/m1.large/ebs/i-ba545fcc/fd0b2556-95c8-4194-a41b-ecb77335c031-terminated.console.txt
ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-3.0.0-9-virtual 3.0.0-9.15
ProcVersionSignature: User Name 3.0.0-9.15-virtual 3.0.3
Uname: Linux 3.0.0-9-virtual x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 2011-08-31 15:34 seq
crw-rw---- 1 root audio 116, 33 2011-08-31 15:34 timer
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
CurrentDmesg: [ 20.210019] eth0: no IPv6 routers present
Date: Wed Aug 31 15:40:06 2011
Ec2AMI: ami-820b38f6
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1b
Ec2InstanceType: m1.large
Ec2Kernel: aki-62695816
Ec2Ramdisk: unavailable
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
ProcEnviron:
PATH=(custom, user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
ProcModules: acpiphp 24080 0 - Live 0x0000000000000000
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install) |
We saw a failure EC2 testing of Beta Candidate for oneiric [1].
In [2], we did collect a 'terminated' log, that shows a the failure.
useful console log at [4], which i'll attach also. The interesting section of log is:
[11963.827784] udevd[82]: starting version 173
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Gave up waiting for root device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Check root= (did the system wait for the right device?)
- Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-label/cloudimg-rootfs does not exist. Dropping to a shell!
BusyBox v1.18.4 (Ubuntu 1:1.18.4-2ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
udevd[91]: timeout: killing '/sbin/blkid -o udev -p /dev/xvda1' [102]
...[repeated]...
udevd[91]: '/sbin/blkid -o udev -p /dev/xvda1' [102] terminated by signal 9 (Killed)
At that point, the test harness rebooted the system and it came back up properly.
This bug was opened on an instance of m1.large in eu-west-1 of the same ami.
[1] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/7/
[2] https://jenkins.qa.ubuntu.com/job/oneiric-server-ec2/7/ARCH=amd64,REGION=eu-west-1,STORAGE=ebs,TEST=multi-part-ud,label=ubuntu-server-ec2-testing/artifact/
ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-3.0.0-9-virtual 3.0.0-9.15
ProcVersionSignature: User Name 3.0.0-9.15-virtual 3.0.3
Uname: Linux 3.0.0-9-virtual x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 2011-08-31 15:34 seq
crw-rw---- 1 root audio 116, 33 2011-08-31 15:34 timer
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
CurrentDmesg: [ 20.210019] eth0: no IPv6 routers present
Date: Wed Aug 31 15:40:06 2011
Ec2AMI: ami-820b38f6
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1b
Ec2InstanceType: m1.large
Ec2Kernel: aki-62695816
Ec2Ramdisk: unavailable
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
ProcEnviron:
PATH=(custom, user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
ProcModules: acpiphp 24080 0 - Live 0x0000000000000000
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install) |
|
