cloud-final is never executed if upstart or any dependency is upgraded during initialization

Bug #1103881 reported by Jean-Baptiste Lallement on 2013-01-24
36
This bug affects 7 people
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Medium
Unassigned
upstart (Ubuntu)
High
Unassigned

Bug Description

During a cloud image initialization, the service cloud-final is never executed if the cloud-config script contains an "upgrade" directive and upstart is part of the upgraded packages:

TESTCASE:
1. Download a cloud image from http://cloud-images.ubuntu.com/raring/ with an outdated version of upstart (e.g 2013.01.23 - I saved this image, if you need it and it is not available from the mirror just ask)
2. Prepare a cloud-config script with the directives (example cloud-config script used by lp:auto-package-testing attached):

apt_update: true
apt_upgrade: true

3. Initialize the cloud image with this cloud-config script

EXPECTED RESULT:
Installation finish

ACTUAL RESULT
The service cloud-final is never executed and the flag boot-finished is not present.

cloud-init logs of a failed installation attached.

Related bugs:
 * bug 1080841: should reload configuration if an upstart job is added
 * bug 1124384: reload-configuration can confuse upstart
 * bug 1163669: cloud-init fails to install juju agent after bootstrap

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: cloud-init (not installed)
ProcVersionSignature: Ubuntu 3.8.0-1.5-generic 3.8.0-rc4
Uname: Linux 3.8.0-1-generic x86_64
ApportVersion: 2.8-0ubuntu2
Architecture: amd64
Date: Thu Jan 24 09:08:19 2013
MarkForUpload: True
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init
UpgradeStatus: Upgraded to raring on 2012-01-31 (358 days ago)

Jean-Baptiste Lallement (jibel) wrote :
Jean-Baptiste Lallement (jibel) wrote :

cloud-config script used by lp:auto-package-testing

James Hunt (jamesodhunt) wrote :

Please can you boot such a system with '--debug' and capture syslog / dmesg / console so we can try and work out what is happening here.

Jean-Baptiste Lallement (jibel) wrote :
Jean-Baptiste Lallement (jibel) wrote :
Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → Medium
Scott Moser (smoser) on 2013-01-28
Changed in upstart (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Scott Moser (smoser) on 2013-02-14
description: updated
Scott Moser (smoser) wrote :

To reproduce outside of "cloud", you can basically follow the blog post that I've got at [1].

Using this user-data to make sure you can login:
#cloud-config-archive
- content: |
   #!/bin/sh
   echo "==== $(date -R): user-script run ===" | tee /run/user-script.log
- filename: myjob.conf
  content: |
   #upstart-job
   description "a test upstart job"
   start on stopped rc RUNLEVEL=[2345]
   console output
   task
   script
   echo "==== $(date -R): upstart job run ===" | tee /run/upstart-job.log
   end script
- content: |
   #cloud-config
   password: passw0rd
   chpasswd: { expire: False }
   ssh_pwauth: True

Then:
sudo apt-get install kvm cloud-utils genisoimage
wget http://cloud-images.ubuntu.com/raring/current/raring-server-cloudimg-amd64-disk1.img -O disk.img.dist
qemu-img convert -O qcow2 disk.img.dist disk.img.orig

qemu-img create -f qcow2 -b disk.img.orig disk.img
cloud-localds my-seed.img my-user-data
kvm -net nic -net user,hostfwd=tcp::2222-:22 \
   -hda disk.img -hdb my-seed.img -m 512

Due to the fact that the bug blocks gettys from running, you'll have to ssh in.
Thats why i've done the host forwarding there.
You'll be able to log in with 'ubuntu' and 'passw0rd'. Remember that console output will be going to the serial device.

--
[1] http://ubuntu-smoser.blogspot.com/2013/02/using-ubuntu-cloud-images-without-cloud.html

Scott Moser (smoser) on 2013-04-03
description: updated
Scott Moser (smoser) wrote :

I've added a simple reproduce for bug 1124384 under lxc . It seems likely to me at least that this bug and that one are related.

Kapil Thangavelu (hazmat) wrote :

i see the issue even without an upstart upgrade.. my upgrade list is. note cloud-init is part of the list.

  apparmor apport apt apt-transport-https apt-utils base-files cloud-init curl
  gcc-4.7-base gir1.2-gudev-1.0 grub-common grub-legacy-ec2 grub-pc
  grub-pc-bin grub2-common kmod language-selector-common libapt-inst1.5
  libapt-pkg4.12 libc-bin libc6 libcurl3 libcurl3-gnutls libgcc1 libglib2.0-0
  libgudev-1.0-0 libisccc90 libkmod2 libpci3 libpython-stdlib libpython2.7
  libpython2.7-minimal libpython2.7-stdlib libpython3-stdlib
  libpython3.3-minimal libpython3.3-stdlib libstdc++6 libudev1 login
  module-init-tools multiarch-support ntpdate passwd pciutils perl perl-base
  perl-modules python python-gi python-minimal python-pycurl
  python-twisted-bin python-twisted-core python2.7 python2.7-minimal python3
  python3-apport python3-distupgrade python3-gi python3-minimal
  python3-problem-report python3-update-manager python3.3 python3.3-minimal
  rsyslog tar ubuntu-minimal ubuntu-release-upgrader-core ubuntu-standard udev
  update-manager-core update-notifier-common

Scott Moser (smoser) wrote :

Kapil,
 your upgrade list includes 'glibc' which is a library dependency of upstart. Because of that, a restart of upstart is necessary.
 (/var/lib/dpkg/info/libc6:amd64.postinst invokes 'telinit u').

Scott Moser (smoser) wrote :

Just for reference 'grep -r "telinit u" /var/lib/dpkg/info' on my system shows:
  libjson0
 libnih1
 libnih-dbus1
 libc6
 libselinux1
 libdbus-1-3
 upstart

summary: - cloud-final is never executed if upstart is upgraded during
- initialization of the image
+ cloud-final is never executed if upstart or any dependency is upgraded
+ during initialization
Matt Rae (mattrae) wrote :

With juju 1.24 this bug appears to be hit on state servers started with 'juju ensure-availability' during cloud-init upgrade, unless enable-os-refresh-update and enable-os-upgrade are set to false.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers