Activity log for bug #1889555

Date Who What changed Old value New value Message
2020-07-30 12:42:13 Dimitri John Ledkov bug added bug
2020-07-30 12:42:28 Dimitri John Ledkov nominated for series Ubuntu Focal
2020-07-30 12:42:28 Dimitri John Ledkov bug task added cloud-init (Ubuntu Focal)
2020-07-30 12:42:28 Dimitri John Ledkov nominated for series Ubuntu Groovy
2020-07-30 12:42:28 Dimitri John Ledkov bug task added cloud-init (Ubuntu Groovy)
2020-07-30 12:42:28 Dimitri John Ledkov nominated for series Ubuntu Xenial
2020-07-30 12:42:28 Dimitri John Ledkov bug task added cloud-init (Ubuntu Xenial)
2020-07-30 12:42:28 Dimitri John Ledkov nominated for series Ubuntu Bionic
2020-07-30 12:42:28 Dimitri John Ledkov bug task added cloud-init (Ubuntu Bionic)
2020-07-30 12:42:36 Dimitri John Ledkov information type Public Public Security
2020-07-30 12:42:46 Dimitri John Ledkov tags regression-update
2020-07-30 13:48:32 Dan Streetman bug added subscriber Dan Streetman
2020-07-30 19:35:08 Dan Watkins cloud-init (Ubuntu Groovy): status New In Progress
2020-07-30 19:35:11 Dan Watkins cloud-init (Ubuntu Groovy): assignee Dan Watkins (oddbloke)
2020-07-30 20:45:29 Robert C Jennings bug added subscriber Robert C Jennings
2020-08-11 06:47:47 Steve Beattie bug added subscriber Steve Beattie
2020-08-28 04:32:22 Chris Halse Rogers cloud-init (Ubuntu Focal): status New Fix Committed
2020-08-28 04:32:24 Chris Halse Rogers bug added subscriber Ubuntu Stable Release Updates Team
2020-08-28 04:32:27 Chris Halse Rogers bug added subscriber SRU Verification
2020-08-28 04:32:34 Chris Halse Rogers tags regression-update regression-update verification-needed verification-needed-focal
2020-08-28 05:45:39 Chris Halse Rogers cloud-init (Ubuntu Bionic): status New Fix Committed
2020-08-28 05:45:49 Chris Halse Rogers tags regression-update verification-needed verification-needed-focal regression-update verification-needed verification-needed-bionic verification-needed-focal
2020-08-28 05:48:40 Chris Halse Rogers cloud-init (Ubuntu Xenial): status New Fix Committed
2020-08-28 05:48:51 Chris Halse Rogers tags regression-update verification-needed verification-needed-bionic verification-needed-focal regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-xenial
2020-09-14 17:03:56 Chad Smith description cc_grub_dpkg was fixed to support nvme drives, but didn't clear the state of cc_grub_dpkg and didn't rerun it on upgrades However, that only fixed the issue for the newly first-booted instances on nvme. All existing boots of cloud-init on nvmes are still broken, and will fail to apply the latest grub2 update for BootHole mitigation. Please add maintainer scripts changes to re-run cc_grub_dpkg, once-only, when cloud-init is upgraded to a new sru. To ensure that cc_grub_dpkg has been rerun, once, since nvme fixes. You could guard this call, if debconf database grub-pc devices do not exist on the instance. (i.e. debconf has /dev/sda, and yet /dev/sda does not exist) === Begin SRU Template === [Impact] Older versions of cloud-init could misconfigure grub on nvme devices, which could prevent instances from booting after a grub upgrade. [Test Case] For focal, bionic, and xenial verify the following: 1. on an affected instance, test that installing the new version of cloud-init appropriately updates debconf 2. on an affected instance, modify of the debconf settings and test that installing the new version of cloud-init does not touch those values 3. in a container, confirm that cloud-init does not touch the values 4. in an unaffected instance (i.e. one without an NVMe root), confirm that cloud-init does not touch the values Steps for test 1: # Find an old affected image with aws ec2 describe-images --filters "Name=name,Values=Ubuntu <release number>*" # Launch an AWS with affected image-id, ssh in # After startup, connect via SSH, then # Verify we're on an nvme device lsblk | grep nvme # Verify install_devices set incorrectly debconf-show grub-pc | grep "install_devices:" # update cloud-init to proposed mirror=http://archive.ubuntu.com/ubuntu echo deb $mirror $(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list apt-get update -q apt-get install -qy cloud-init # Verify "Reconfiguring grub" message in upgrade output # Verify install_devices set correctly debconf-show grub-pc | grep "install_devices:" # Verify that after reboot we can still connect Steps for test 2: # Find an old affected image with aws ec2 describe-images --filters "Name=name,Values=Ubuntu <release number>*" # Launch an AWS with affected image-id # After startup, connect via SSH, then # Verify we're on an nvme device lsblk | grep nvme # Verify install_devices set incorrectly debconf-show grub-pc | grep "install_devices:" # Update install device to something (anything) else echo 'set grub-pc/install_devices /dev/sdb' | debconf-communicate # update cloud-init to proposed mirror=http://archive.ubuntu.com/ubuntu echo deb $mirror $(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list apt-get update -q apt-get install -qy cloud-init # Verify no "Reconfiguring grub" message in upgrade output # Verify install_devices not changed debconf-show grub-pc | grep "install_devices:" Steps for test 3: # lxd launch affected image lxc launch <image> # Obtain bash shell lxc exec <image> bash # Check install_devices debconf-show grub-pc | grep "install_devices:" # Update cloud-init to propsed mirror=http://archive.ubuntu.com/ubuntu echo deb $mirror $(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list apt-get update -q apt-get install -qy cloud-init # Verify no "Reconfiguring grub" message in upgrade output # Verify install_devices not changed debconf-show grub-pc | grep "install_devices:" Steps for test 4: # Launch GCE image with: gcloud compute instances create falcon-test --image <image> --image-project ubuntu-os-cloud --zone=us-central1-a # After startup, connect via SSH, then # Verify we're not on an nvme device lsblk | grep nvme # Check install_devices debconf-show grub-pc | grep "install_devices:" # update cloud-init to proposed # Verify "Reconfiguring grub" message not in upgrade output # Verify install_devices set correctly debconf-show grub-pc | grep "install_devices:" # Verify that after reboot we can still connect [Regression Potential] If a user manually configured their system in such a way that both devices exist and it matches our error condition, the grub install device could be reconfigured incorrectly. [Other Info] Pull request: https://github.com/canonical/cloud-init/pull/514/files Upstream commit: https://github.com/canonical/cloud-init/commit/f48acc2bdc41c347d2eb899038e2520383851103 ==== Original Description ==== cc_grub_dpkg was fixed to support nvme drives, but didn't clear the state of cc_grub_dpkg and didn't rerun it on upgrades However, that only fixed the issue for the newly first-booted instances on nvme. All existing boots of cloud-init on nvmes are still broken, and will fail to apply the latest grub2 update for BootHole mitigation. Please add maintainer scripts changes to re-run cc_grub_dpkg, once-only, when cloud-init is upgraded to a new sru. To ensure that cc_grub_dpkg has been rerun, once, since nvme fixes. You could guard this call, if debconf database grub-pc devices do not exist on the instance. (i.e. debconf has /dev/sda, and yet /dev/sda does not exist)
2020-09-14 17:59:06 James Falcon attachment added Manual verification of issue https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1889555/+attachment/5410817/+files/lp-1889555.txt
2020-09-14 17:59:52 James Falcon tags regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-xenial regression-update verification-done verification-done-bionic verification-done-focal verification-done-xenial
2020-09-16 04:35:45 Launchpad Janitor cloud-init (Ubuntu Groovy): status In Progress Fix Released
2020-09-16 06:51:37 Chris Halse Rogers removed subscriber Ubuntu Stable Release Updates Team
2020-09-16 06:52:00 Launchpad Janitor cloud-init (Ubuntu Focal): status Fix Committed Fix Released
2020-09-16 06:52:19 Launchpad Janitor cloud-init (Ubuntu Bionic): status Fix Committed Fix Released
2020-09-16 06:53:06 Launchpad Janitor cloud-init (Ubuntu Xenial): status Fix Committed Fix Released