Activity log for bug #1693361

Date Who What changed Old value New value Message
2017-05-24 21:10:37 Jim Browne bug added bug
2017-06-06 03:20:52 David Duncan bug added subscriber David Duncan
2017-06-11 12:19:13 Dominique Poulain bug added subscriber Dominique Poulain
2017-06-12 16:59:55 Robie Basak bug added subscriber Robie Basak
2017-06-12 17:03:41 David Britton cloud-init: importance Undecided High
2017-06-12 17:29:33 David Britton bug task added cloud-init (Ubuntu)
2017-06-12 17:29:56 David Britton nominated for series Ubuntu Artful
2017-06-12 17:29:56 David Britton nominated for series Ubuntu Xenial
2017-06-12 17:55:11 Scott Moser nominated for series Ubuntu Zesty
2017-06-12 17:55:11 Scott Moser bug task added cloud-init (Ubuntu Zesty)
2017-06-12 17:55:11 Scott Moser nominated for series Ubuntu Yakkety
2017-06-12 17:55:11 Scott Moser bug task added cloud-init (Ubuntu Yakkety)
2017-06-12 17:55:19 Scott Moser bug task added cloud-init (Ubuntu Artful)
2017-06-12 17:55:25 Scott Moser bug task added cloud-init (Ubuntu Xenial)
2017-06-12 17:56:13 Scott Moser cloud-init: status New Confirmed
2017-06-12 17:56:16 Scott Moser cloud-init: importance High Medium
2017-06-12 17:56:19 Scott Moser cloud-init (Ubuntu Xenial): status New Confirmed
2017-06-12 17:56:22 Scott Moser cloud-init (Ubuntu Yakkety): status New Confirmed
2017-06-12 17:56:24 Scott Moser cloud-init (Ubuntu Zesty): status New Confirmed
2017-06-12 17:56:27 Scott Moser cloud-init (Ubuntu Artful): status New Confirmed
2017-06-12 17:56:30 Scott Moser cloud-init (Ubuntu Xenial): importance Undecided Medium
2017-06-12 17:56:32 Scott Moser cloud-init (Ubuntu Yakkety): importance Undecided Medium
2017-06-12 17:56:34 Scott Moser cloud-init (Ubuntu Zesty): importance Undecided Medium
2017-06-12 17:56:36 Scott Moser cloud-init (Ubuntu Artful): importance Undecided High
2017-06-12 17:56:41 Scott Moser cloud-init (Ubuntu Artful): importance High Medium
2017-06-12 18:20:57 Launchpad Janitor merge proposal linked https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/325513
2017-06-12 19:00:03 David Britton bug task added apt (Ubuntu)
2017-06-12 19:00:37 David Britton bug task deleted apt (Ubuntu Zesty)
2017-06-12 19:00:43 David Britton bug task deleted apt (Ubuntu Yakkety)
2017-06-12 19:00:50 David Britton bug task deleted apt (Ubuntu Xenial)
2017-06-12 19:01:02 David Britton bug task deleted apt (Ubuntu Artful)
2017-06-12 20:19:56 David Britton bug watch added http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=864681
2017-06-12 20:19:56 David Britton bug task added apt
2017-06-13 03:05:20 Bug Watch Updater apt: status Unknown New
2017-06-14 07:25:40 Bug Watch Updater apt: status New Confirmed
2017-06-20 20:31:34 David Britton cloud-init (Ubuntu Artful): status Confirmed Fix Committed
2017-06-27 21:56:55 Launchpad Janitor cloud-init (Ubuntu Artful): status Fix Committed Fix Released
2017-06-28 17:56:59 Launchpad Janitor merge proposal linked https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/326452
2017-06-28 21:15:57 Scott Moser description apt-daily is now a systemd service rather than being invoked by cron.daily. If one builds a custom AMI it is possible that the apt-daily.timer will fire during boot. This can fire at the same time cloud-init is running and if cloud-init loses the race the invocation of apt (e.g. use of "packages:" in the config) will fail. There is a lot of discussion online about this change to apt-daily (e.g. unattended upgrades happening during business hours, delaying boot, etc.) and discussion of potential systemd changes regarding timers firing during boot (c.f. https://github.com/systemd/systemd/issues/5659). While it would be better to solve this in apt itself, I suggest that cloud-init be defensive when calling apt and implement some retry mechanism. Various instances of people running into this issue: https://github.com/chef/bento/issues/609 https://clusterhq.atlassian.net/browse/FLOC-4486 https://github.com/boxcutter/ubuntu/issues/73 https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image === Begin SRU Template === [Impact] A cloud-config that contains packages to install (see below) or 'package_upgrade' will run 'apt-get update'. That can sometimes fail as a result of contention with the apt-daily.service that updates that information. Cloud-config showing the problem is just like: $ cat my.yaml #cloud-config packages: ['hello'] [Test Case] lxc-proposed-snapshot is https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot It publishes an image to lxd with proposed enabled and cloud-init upgraded. a.) launch an instance with proposed version of cloud-init and some user-data. This is platform independent. The test case demonstrates lxd. $ printf "%s\n%s\n%s\n" "#cloud-config" "packages: ['hello']" \ "package_upgrade: true" > config.yaml $ release=xenial $ ref=proposed-$release    $ ./lxc-proposed-snapshot --proposed --publish $release $ref; b.) start the instance $ name=$release-1693361 $ lxc launch my-xenial "--config=user.user-data=$(cat config.yaml)    $ sleep 1 $ lxc exec $name -- tail -f /var/log/cloud-init.log /var/log/cloud-init-output.log # watch this boot. c.) Look for evidence of systemd failure journalctl -o short-precise | grep -i break journalctl -o short-precise | grep -i order [Regression Potential] Regression chance here is low. Its possible that ordering dependencies could occur. When that does happen, journalctl will mention it. Unfortunately in such cases systemd somewhat randomly picks a service to kil so behavior is somewhat undefined. [Other Info] Upstream commit at https://git.launchpad.net/cloud-init/commit/?id=11121fe4 === End SRU Template === apt-daily is now a systemd service rather than being invoked by cron.daily. If one builds a custom AMI it is possible that the apt-daily.timer will fire during boot. This can fire at the same time cloud-init is running and if cloud-init loses the race the invocation of apt (e.g. use of "packages:" in the config) will fail. There is a lot of discussion online about this change to apt-daily (e.g. unattended upgrades happening during business hours, delaying boot, etc.) and discussion of potential systemd changes regarding timers firing during boot (c.f. https://github.com/systemd/systemd/issues/5659). While it would be better to solve this in apt itself, I suggest that cloud-init be defensive when calling apt and implement some retry mechanism. Various instances of people running into this issue: https://github.com/chef/bento/issues/609 https://clusterhq.atlassian.net/browse/FLOC-4486 https://github.com/boxcutter/ubuntu/issues/73 https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image
2017-06-29 00:28:25 Steve Langasek description === Begin SRU Template === [Impact] A cloud-config that contains packages to install (see below) or 'package_upgrade' will run 'apt-get update'. That can sometimes fail as a result of contention with the apt-daily.service that updates that information. Cloud-config showing the problem is just like: $ cat my.yaml #cloud-config packages: ['hello'] [Test Case] lxc-proposed-snapshot is https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot It publishes an image to lxd with proposed enabled and cloud-init upgraded. a.) launch an instance with proposed version of cloud-init and some user-data. This is platform independent. The test case demonstrates lxd. $ printf "%s\n%s\n%s\n" "#cloud-config" "packages: ['hello']" \ "package_upgrade: true" > config.yaml $ release=xenial $ ref=proposed-$release    $ ./lxc-proposed-snapshot --proposed --publish $release $ref; b.) start the instance $ name=$release-1693361 $ lxc launch my-xenial "--config=user.user-data=$(cat config.yaml)    $ sleep 1 $ lxc exec $name -- tail -f /var/log/cloud-init.log /var/log/cloud-init-output.log # watch this boot. c.) Look for evidence of systemd failure journalctl -o short-precise | grep -i break journalctl -o short-precise | grep -i order [Regression Potential] Regression chance here is low. Its possible that ordering dependencies could occur. When that does happen, journalctl will mention it. Unfortunately in such cases systemd somewhat randomly picks a service to kil so behavior is somewhat undefined. [Other Info] Upstream commit at https://git.launchpad.net/cloud-init/commit/?id=11121fe4 === End SRU Template === apt-daily is now a systemd service rather than being invoked by cron.daily. If one builds a custom AMI it is possible that the apt-daily.timer will fire during boot. This can fire at the same time cloud-init is running and if cloud-init loses the race the invocation of apt (e.g. use of "packages:" in the config) will fail. There is a lot of discussion online about this change to apt-daily (e.g. unattended upgrades happening during business hours, delaying boot, etc.) and discussion of potential systemd changes regarding timers firing during boot (c.f. https://github.com/systemd/systemd/issues/5659). While it would be better to solve this in apt itself, I suggest that cloud-init be defensive when calling apt and implement some retry mechanism. Various instances of people running into this issue: https://github.com/chef/bento/issues/609 https://clusterhq.atlassian.net/browse/FLOC-4486 https://github.com/boxcutter/ubuntu/issues/73 https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image === Begin SRU Template === [Impact] A cloud-config that contains packages to install (see below) or 'package_upgrade' will run 'apt-get update'. That can sometimes fail as a result of contention with the apt-daily.service that updates that information. Cloud-config showing the problem is just like:   $ cat my.yaml   #cloud-config   packages: ['hello'] [Test Case] lxc-proposed-snapshot is   https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot It publishes an image to lxd with proposed enabled and cloud-init upgraded. a.) launch an instance with proposed version of cloud-init and some user-data.    This is platform independent. The test case demonstrates lxd.    $ printf "%s\n%s\n%s\n" "#cloud-config" "packages: ['hello']" \        "package_upgrade: true" > config.yaml    $ release=xenial    $ ref=proposed-$release    $ ./lxc-proposed-snapshot --proposed --publish $release $ref; b.) start the instance    $ name=$release-1693361    $ lxc launch my-xenial "--config=user.user-data=$(cat config.yaml)    $ sleep 1    $ lxc exec $name -- tail -f /var/log/cloud-init.log /var/log/cloud-init-output.log    # watch this boot.  c.) Look for evidence of systemd failure    journalctl -o short-precise | grep -i break    journalctl -o short-precise | grep -i order [Regression Potential] Regression chance here is low. Its possible that ordering loops could occur. When that does happen, journalctl will mention it. Unfortunately in such cases systemd somewhat randomly picks a service to kil so behavior is somewhat undefined. [Other Info] Upstream commit at   https://git.launchpad.net/cloud-init/commit/?id=11121fe4 === End SRU Template === apt-daily is now a systemd service rather than being invoked by cron.daily. If one builds a custom AMI it is possible that the apt-daily.timer will fire during boot. This can fire at the same time cloud-init is running and if cloud-init loses the race the invocation of apt (e.g. use of "packages:" in the config) will fail. There is a lot of discussion online about this change to apt-daily (e.g. unattended upgrades happening during business hours, delaying boot, etc.) and discussion of potential systemd changes regarding timers firing during boot (c.f. https://github.com/systemd/systemd/issues/5659). While it would be better to solve this in apt itself, I suggest that cloud-init be defensive when calling apt and implement some retry mechanism. Various instances of people running into this issue: https://github.com/chef/bento/issues/609 https://clusterhq.atlassian.net/browse/FLOC-4486 https://github.com/boxcutter/ubuntu/issues/73 https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image
2017-06-29 04:45:22 Steve Langasek cloud-init (Ubuntu Zesty): status Confirmed Fix Committed
2017-06-29 04:45:24 Steve Langasek bug added subscriber Ubuntu Stable Release Updates Team
2017-06-29 04:45:26 Steve Langasek bug added subscriber SRU Verification
2017-06-29 04:45:31 Steve Langasek tags verification-needed verification-needed-zesty
2017-06-29 04:53:08 Steve Langasek cloud-init (Ubuntu Yakkety): status Confirmed Fix Committed
2017-06-29 04:53:15 Steve Langasek tags verification-needed verification-needed-zesty verification-needed verification-needed-yakkety verification-needed-zesty
2017-06-29 04:55:47 Steve Langasek cloud-init (Ubuntu Xenial): status Confirmed Fix Committed
2017-06-29 04:55:55 Steve Langasek tags verification-needed verification-needed-yakkety verification-needed-zesty verification-needed verification-needed-xenial verification-needed-yakkety verification-needed-zesty
2017-06-29 19:22:00 Scott Moser attachment added xenial results https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1693361/+attachment/4905922/+files/1693361-xenial.tar.gz
2017-06-29 19:22:19 Scott Moser attachment added yakkety results https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1693361/+attachment/4905923/+files/1693361-yakkety.tar.gz
2017-06-29 19:22:45 Scott Moser attachment added zesty results https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1693361/+attachment/4905924/+files/1693361-zesty.tar.gz
2017-06-29 19:23:34 Scott Moser tags verification-needed verification-needed-xenial verification-needed-yakkety verification-needed-zesty verification-done-xenial verification-done-yakkety verification-done-zesty
2017-06-29 21:48:59 Launchpad Janitor cloud-init (Ubuntu Xenial): status Fix Committed Fix Released
2017-06-29 21:49:10 Steve Langasek removed subscriber Ubuntu Stable Release Updates Team
2017-07-26 18:27:42 Launchpad Janitor cloud-init (Ubuntu Zesty): status Fix Committed Fix Released
2017-07-26 18:28:30 Steve Langasek cloud-init (Ubuntu Yakkety): status Fix Committed Won't Fix
2017-09-23 02:33:25 Scott Moser cloud-init: status Confirmed Fix Released
2018-08-22 10:54:05 Julian Andres Klode apt (Ubuntu): status New Invalid
2020-04-09 12:04:28 Bug Watch Updater apt: status Confirmed Fix Released
2021-11-12 12:46:19 David Reis bug added subscriber David Reis
2022-04-13 17:30:24 Jesús Gómez bug added subscriber Jesús Gómez
2022-04-13 17:38:06 Jesús Gómez attachment added cloud-init.tar.gz https://bugs.launchpad.net/cloud-init/+bug/1693361/+attachment/5580471/+files/cloud-init.tar.gz
2023-04-02 02:39:32 Nathan A. Ferch bug added subscriber Nathan A. Ferch
2023-05-11 02:49:40 James Falcon bug watch added https://github.com/canonical/cloud-init/issues/2908
2024-07-23 23:35:41 Richard Mansfield bug added subscriber Richard Mansfield