t1.micro instance defect after dist-upgrade

Bug #927805 reported by Hinnerk
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Expired
Medium
Unassigned

Bug Description

I verified this twice: after dis-upgrade the system won't fully come back. Pink works but ssh login doesn't.

apt-get update; apt-get upgrade; reboot

apt-get dist-upgrade; reboot

There where quite a few error messages during dist-upgrade:

# apt-get dist-upgrade
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut
Status-Informationen einlesen... Fertig
Berechne Upgrade (Paketaktualisierung) ...Fertig
Die folgenden NEUEN Pakete werden installiert:
  linux-image-2.6.32-342-ec2 linux-image-2.6.32-38-virtual
Die folgenden Pakete werden aktualisiert:
  linux-ec2 linux-image-ec2 linux-image-virtual linux-virtual
4 aktualisiert, 2 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 31,5MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 85,7MB Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren [J/n]?
Hole:1 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-image-2.6.32-342-ec2 2.6.32-342.43 [19,4MB]
Hole:2 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-image-2.6.32-38-virtual 2.6.32-38.83 [12,1MB]
Hole:3 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-ec2 2.6.32.342.23 [3.626B]
Hole:4 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-image-ec2 2.6.32.342.23 [3.634B]
Hole:5 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-virtual 2.6.32.38.44 [4.774B]
Hole:6 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates/main linux-image-virtual 2.6.32.38.44 [4.778B]
Es wurden 31,5MB in 1s geholt (28,5MB/s)
Wähle vormals abgewähltes Paket linux-image-2.6.32-342-ec2.
(Lese Datenbank ... 42298 Dateien und Verzeichnisse sind derzeit installiert.)
Entpacke linux-image-2.6.32-342-ec2 (aus .../linux-image-2.6.32-342-ec2_2.6.32-342.43_i386.deb) ...
Done.
Wähle vormals abgewähltes Paket linux-image-2.6.32-38-virtual.
Entpacke linux-image-2.6.32-38-virtual (aus .../linux-image-2.6.32-38-virtual_2.6.32-38.83_i386.deb) ...
Done.
Vorbereiten zum Ersetzen von linux-ec2 2.6.32.318.19 (durch .../linux-ec2_2.6.32.342.23_i386.deb) ...
Entpacke Ersatz für linux-ec2 ...
Vorbereiten zum Ersetzen von linux-image-ec2 2.6.32.318.19 (durch .../linux-image-ec2_2.6.32.342.23_i386.deb) ...
Entpacke Ersatz für linux-image-ec2 ...
Vorbereiten zum Ersetzen von linux-virtual 2.6.32.34.40 (durch .../linux-virtual_2.6.32.38.44_i386.deb) ...
Entpacke Ersatz für linux-virtual ...
Vorbereiten zum Ersetzen von linux-image-virtual 2.6.32.34.40 (durch .../linux-image-virtual_2.6.32.38.44_i386.deb) ...
Entpacke Ersatz für linux-image-virtual ...
Richte linux-image-2.6.32-342-ec2 ein (2.6.32-342.43) ...
Running depmod.
update-initramfs: Generating /boot/initrd.img-2.6.32-342-ec2
Running postinst hook script /usr/sbin/update-grub.
head: „/boot/grub/video.lst“ kann nicht zum Lesen geöffnet werden: Datei oder Verzeichnis nicht gefunden
Generating grub.cfg ...
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-312-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-314-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-317-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-318-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-342-ec2
Found linux image: /boot/vmlinuz-2.6.32-38-generic-pae
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
Found linux image: /boot/vmlinuz-2.6.32-34-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-34-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-33-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-33-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-32-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-32-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-31-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-31-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-30-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-30-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-28-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-28-generic-pae
Found memtest86+ image: /boot/memtest86+.bin
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
done
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.32-342-ec2 /boot/vmlinuz-2.6.32-342-ec2
run-parts: executing /etc/kernel/postinst.d/x-grub-legacy-ec2 2.6.32-342-ec2 /boot/vmlinuz-2.6.32-342-ec2
Searching for GRUB installation directory ... found: /boot/grub
Searching for default file ... found: /boot/grub/default
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-38-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-34-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-33-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-32-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-31-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-30-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-28-generic-pae
Found kernel: /boot/vmlinuz-2.6.32-342-ec2
Found kernel: /boot/vmlinuz-2.6.32-318-ec2
Found kernel: /boot/vmlinuz-2.6.32-317-ec2
Found kernel: /boot/vmlinuz-2.6.32-314-ec2
Found kernel: /boot/vmlinuz-2.6.32-312-ec2
Found kernel: /boot/memtest86+.bin
Replacing config file /var/run/grub/menu.lst with new version
Updating /boot/grub/menu.lst ... done

Richte linux-image-2.6.32-38-virtual ein (2.6.32-38.83) ...
Running depmod.
update-initramfs: Generating /boot/initrd.img-2.6.32-38-generic-pae
Running postinst hook script /usr/sbin/update-grub.
head: „/boot/grub/video.lst“ kann nicht zum Lesen geöffnet werden: Datei oder Verzeichnis nicht gefunden
Generating grub.cfg ...
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-312-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-314-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-317-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-318-ec2
Skipping linux image [linux-ec2]: /boot/vmlinuz-2.6.32-342-ec2
Found linux image: /boot/vmlinuz-2.6.32-38-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-38-generic-pae
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
Found linux image: /boot/vmlinuz-2.6.32-34-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-34-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-33-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-33-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-32-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-32-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-31-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-31-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-30-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-30-generic-pae
Found linux image: /boot/vmlinuz-2.6.32-28-generic-pae
Found initrd image: /boot/initrd.img-2.6.32-28-generic-pae
Found memtest86+ image: /boot/memtest86+.bin
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
/usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/sda1. Check your device.map.
done
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.32-38-generic-pae /boot/vmlinuz-2.6.32-38-generic-pae
run-parts: executing /etc/kernel/postinst.d/x-grub-legacy-ec2 2.6.32-38-generic-pae /boot/vmlinuz-2.6.32-38-generic-pae
Searching for GRUB installation directory ... found: /boot/grub
Searching for default file ... found: /boot/grub/default
Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst
Searching for splash image ... none found, skipping ...
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-38-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-34-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-33-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-32-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-31-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-30-generic-pae
Ignoring non-Xen Kernel on Xen domU host: vmlinuz-2.6.32-28-generic-pae
Found kernel: /boot/vmlinuz-2.6.32-342-ec2
Found kernel: /boot/vmlinuz-2.6.32-318-ec2
Found kernel: /boot/vmlinuz-2.6.32-317-ec2
Found kernel: /boot/vmlinuz-2.6.32-314-ec2
Found kernel: /boot/vmlinuz-2.6.32-312-ec2
Found kernel: /boot/memtest86+.bin
Updating /boot/grub/menu.lst ... done

Richte linux-image-ec2 ein (2.6.32.342.23) ...
Richte linux-ec2 ein (2.6.32.342.23) ...
Richte linux-image-virtual ein (2.6.32.38.44) ...
Richte linux-virtual ein (2.6.32.38.44) ...

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: grub-legacy-ec2 0.5.10-0ubuntu1.5
ProcVersionSignature: Ubuntu 2.6.32-318.38-ec2 2.6.32.44+drm33.19
Uname: Linux 2.6.32-318-ec2 i686
Architecture: i386
Date: Mon Feb 6 18:55:50 2012
Ec2AMI: ami-e398aa97
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1a
Ec2InstanceType: t1.micro
Ec2Kernel: aki-4deec439
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 LC_CTYPE=de_DE.UTF-8
 PATH=(custom, no user)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init

Revision history for this message
Hinnerk (haardt) wrote :
Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Can you please post a copy of the EC2 console.

Revision history for this message
Hinnerk (haardt) wrote :
Download full text (13.0 KiB)

Yes but I guess it's the one from the old kernel.

  Xen Minimal OS!
  start_info: 0xa01000(VA)
    nr_pages: 0x26700
  shared_inf: 0x98035000(MA)
     pt_base: 0xa04000(VA)
nr_pt_frames: 0x9
    mfn_list: 0x967000(VA)
   mod_start: 0x0(VA)
     mod_len: 0
       flags: 0x0
    cmd_line: root=/dev/sda1 ro 4
  stack: 0x946780-0x966780
MM: Init
      _text: 0x0(VA)
     _etext: 0x621f5(VA)
   _erodata: 0x76000(VA)
     _edata: 0x7b6d4(VA)
stack start: 0x946780(VA)
       _end: 0x966d34(VA)
  start_pfn: a10
    max_pfn: 26700
Mapping memory range 0xc00000 - 0x26700000
setting 0x0-0x76000 readonly
skipped 0x1000
MM: Initialise page allocator for b3e000(b3e000)-0(26700000)
MM: done
Demand map pfns at 26701000-36701000.
Heap resides at 36702000-76702000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x26701000.
Initialising scheduler
Thread "Idle": pointer: 0x36702008, stack: 0xbf0000
Initialising xenbus
Thread "xenstore": pointer: 0x36702478, stack: 0x26600000
Dummy main: start_info=0x966880
Thread "main": pointer: 0x367028e8, stack: 0x26610000
"main" "root=/dev/sda1" "ro" "4"
vbd 2049 is hd0
******************* BLKFRONT for device/vbd/2049 **********

backend at /local/domain/0/backend/vbd/346/2049
Failed to read /local/domain/0/backend/vbd/346/2049/feature-barrier.
Failed to read /local/domain/0/backend/vbd/346/2049/feature-flush-cache.
16777216 sectors of 0 bytes
**************************

    [H
    [J Booting 'Ubuntu 10.04.3 LTS, kernel 2.6.32-318-ec2'

root (hd0)

 Filesystem type is ext2fs, using whole disk

kernel /boot/vmlinuz-2.6.32-318-ec2 root=LABEL=uec-rootfs ro xencons=hvc0 cons

ole=hvc0

initrd /boot/initrd.img-2.6.32-318-ec2

xc_dom_probe_bzimage_kernel: kernel is not a bzImage
close blk: backend at /local/domain/0/backend/vbd/346/2049
[ 0.000000] Reserving virtual address space above 0xf5800000

[ 0.000000] Initializing cgroup subsys cpuset

[ 0.000000] Initializing cgroup subsys cpu

[ 0.000000] Linux version 2.6.32-318-ec2 (buildd@roseapple) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #38-Ubuntu SMP Thu Sep 1 17:54:33 UTC 2011 (Ubuntu 2.6.32-318.38-ec2 2.6.32.44+drm33.19)

[ 0.000000] KERNEL supported cpus:

[ 0.000000] Intel GenuineIntel

[ 0.000000] AMD AuthenticAMD

[ 0.000000] NSC Geode by NSC

[ 0.000000] Cyrix CyrixInstead

[ 0.000000] Centaur CentaurHauls

[ 0.000000] Transmeta GenuineTMx86

[ 0.000000] Transmeta TransmetaCPU

[ 0.000000] Xen-provided physical RAM map:

[ 0.000000] Xen: 0000000000000000 - 0000000026f00000 (usable)

[ 0.000000] last_pfn = 0x26f00 max_arch_pfn = 0x10000000

[ 0.000000] init_memory_mapping: 0000000000000000-0000000026f00000

[ 0.000000] NX (Execute Disable) protection: active

[ 0.000000] RAMDISK: 007f4000 - 017d2000

[ 0.000000] 0MB HIGHMEM available.

[ 0.000000] 623MB LOWMEM available.

[ 0.000000] mapped low ram: 0 - 26f00000

[ 0.000000] low ram: 0 - 26f00000

[ 0.000000] node 0 low ram: 00000000 - 26700000

[ 0.000000] node 0 bootmap 00000000 - 00004ce0

[ 0.000000] (4 early reservations) ==> bootmem [0000000000 - 0026700000]

[ ...

Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Also, can you post the region and the AMI that you were using.

I was unable to confirm with the latest EBS AMI.

Changed in cloud-init (Ubuntu):
status: New → Incomplete
Revision history for this message
Hinnerk (haardt) wrote :

The system was built around 05/2011 and until 10/2011 updated at least monthly, each update involving a rebuild of the AMI. I currently can't see which one was the original AMI from Ubuntu.

The Region is eu-west-1a.

Robie Basak (racb)
Changed in cloud-init (Ubuntu):
status: Incomplete → New
Changed in cloud-init (Ubuntu):
importance: Undecided → High
Revision history for this message
Scott Moser (smoser) wrote :

Hi,
  I'm sorry, but there is just not enough information to make any sort of diagnosis on what caused the issue.
  The 'grub-probe' error messages in the upgrade can be ignored. They're there only because grub-pc package is installed, but it is not being used. The pv-grub aki from amazon is loading the image.

  Can you please collect a get-console-output of a failed instance?
  Alternatively, you are welcome to give me access to the ami that you upgraded and failed, and I can try to look.

  Without one of those bits of information, we really can't make any progress on this bug. As Ben noted, the issue does not occur on current AMIs. Without being able to reproduce or see information on a failed state, we simply cannot help.

  I've tried to reproduce by launching a t1.micro instance of 'ami-311f2b45'
(eu-west-1 ebs/ubuntu-lucid-10.04-i386-server-20110201.1). That is the oldest ami available that uses the pv-grub kernel you reported this issue on (aki-4deec439).

  I then ran 'sudo apt-get update && sudo apt-get dist-upgrade'. After that was successfull, I rebooted. The system came up fine into the kernel 'vmlinuz-2.6.32-342-ec2' (which is shown in your output).

Changed in cloud-init (Ubuntu):
importance: High → Medium
status: New → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for cloud-init (Ubuntu) because there has been no activity for 60 days.]

Changed in cloud-init (Ubuntu):
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.