cloud-init status --wait never returns if datasource not found

Bug #1966085 reported by James Falcon
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
James Falcon
cloud-init (Ubuntu)
Fix Released
High
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
Unassigned
Impish
Fix Released
Undecided
Unassigned
Jammy
Fix Released
High
Unassigned

Bug Description

[Impact]

 * In environments where cloud-init-generator detects no viable datasources, cloud-init systemd units are not included in the boot process and `cloud-init status --wait` blocks indefinitely reporting "not run" state instead of "disabled". Utilities which rely on the blocking status will sit indefinitely as cloud-init will never run in these scenarios.

This upload ensures that the expected artifact of cloud-init-generator emits a /run/cloud-init/disabled file (which cloud-init status --wait expects to see when coud-init is disabled at generator timeframe) which allows cloud-init status --wait to return expected "disabled" status.

[Test Plan]

 * this scenario is reproducible by disabling LXD nocloud metadata templates and security.devlxd=false configuration which will allow cloud-init-generator to find no viable datasources during systemd generator time frame. Cloud-init status --wait in this case should not block, but return 'disabled' status immediately after systemd generator is run.

Procedure
 * lxc launch ubuntu-daily:<series> test-series
 # To avoid LXD datasource detection don't allow /dev/lxd/sock in instance
 * lxc config set test-<series< security.devlxd=false
 # Remove nocloud-net cloud-init instance metadata to avoid NoCloud
 * lxc exec test-<series> -- rm -rf /var/lib/cloud/seed/nocloud-net
 # clean reboot of cloud-init to ensure generator detection
 * lxc exec test-<series> -- cloud-init clean --logs --reboot

 # make sure cloud-init status --wait doesn't block indefiinitely in this case

 * lxc exec tests-<series> -- cloud-init status --wait

 This test procedure is captured by cloud-init integration test
https://github.com/canonical/cloud-init/blob/main/tests/integration_tests/cmd/test_status.py

It can be run with the following:

for $SERIES in bionic focal impish; do
  CLOUD_INIT_PLATFORM=lxd_container CLOUD_INIT_CLOUD_INIT_SOURCE=PROPOSED CLOUD_INIT_OS_IMAGE=$SERIES tox -e integration-tests tests/integration_tests/cmd/test_status.py
done

[Where problems could occur]

 * Problems can occur if cloud-init early exits from `cloud-init status --wait` without blocking if 3rd party apps/scripts which expect to block until cloud-init completes start running before cloud-init is "done" could could lead to unexpected system configuration or apt-lock issues if scripts attempt to install packages

[Other Info]

[Original Description]

Calling "cloud-init status --wait" never returns if ds-identify cannot find a datasource.

In https://github.com/canonical/cloud-init/pull/1162, we modified status checks to wait until we get an "enabled" or "disabled" file from ds-identiy. ds-identify never outputs a "disabled" file, so "status --wait" will wait indefinitely if no datasource is found.

A simple reproduction is to change "ret=1" on this line:
https://github.com/canonical/cloud-init/blob/main/tools/ds-identify#L1866

James Falcon (falcojr)
Changed in cloud-init:
status: New → Triaged
importance: Undecided → High
Changed in cloud-init (Ubuntu):
importance: Undecided → High
status: New → Triaged
James Falcon (falcojr)
description: updated
tags: added: regression-update
Chad Smith (chad.smith)
Changed in cloud-init (Ubuntu Bionic):
status: New → In Progress
Changed in cloud-init (Ubuntu Focal):
status: New → In Progress
Changed in cloud-init (Ubuntu Impish):
status: New → In Progress
Changed in cloud-init (Ubuntu Jammy):
status: Triaged → In Progress
Changed in cloud-init:
assignee: nobody → James Falcon (falcojr)
Chad Smith (chad.smith)
Changed in cloud-init:
status: Triaged → In Progress
Revision history for this message
Chad Smith (chad.smith) wrote :
Revision history for this message
Chad Smith (chad.smith) wrote :
Changed in cloud-init:
status: In Progress → Fix Committed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Backports to Ubuntu releases started for Jammy at https://github.com/canonical/cloud-init/pull/1350

Revision history for this message
Chad Smith (chad.smith) wrote :

Fix released to Jammy [ubuntu/jammy-proposed] cloud-init 22.1-14-g2e17a0d6-0ubuntu1~22.04.4 (Accepted)

Changed in cloud-init (Ubuntu Jammy):
status: In Progress → Fix Released
Changed in cloud-init:
status: Fix Committed → Fix Released
Chad Smith (chad.smith)
description: updated
Chad Smith (chad.smith)
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello James, or anyone else affected,

Accepted cloud-init into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.1-14-g2e17a0d6-0ubuntu1~21.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-init (Ubuntu Impish):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-impish
Changed in cloud-init (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello James, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.1-14-g2e17a0d6-0ubuntu1~20.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-init (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello James, or anyone else affected,

Accepted cloud-init into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.1-14-g2e17a0d6-0ubuntu1~18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Chad Smith (chad.smith) wrote :

Uploads queued for -proposed testing on bionic focal and impish: verification will start once -proposed are published and accessible

$ rmadison cloud-init | grep proposed
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~18.04.2 | bionic-proposed | all
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~18.04.3 | bionic-proposed | source
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~20.04.2 | focal-proposed | all
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~20.04.3 | focal-proposed | source
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~21.10.2 | impish-proposed | all
 cloud-init | 22.1-14-g2e17a0d6-0ubuntu1~21.10.3 | impish-proposed | source

Chad Smith (chad.smith)
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
James Falcon (falcojr) wrote (last edit ):

Verification on Bionic

Revision history for this message
James Falcon (falcojr) wrote (last edit ):

Verification on Focal

Revision history for this message
James Falcon (falcojr) wrote (last edit ):

Verification on Impish

tags: added: verification-done-bionic verification-done-focal verification-done-impish
removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-impish
Revision history for this message
Chad Smith (chad.smith) wrote :

Supplemental full integration test suite on bionic lxd-container to ensure no other functionality broke

Revision history for this message
Chad Smith (chad.smith) wrote :

sru-22.1-lxd-container-focal.log SUCCESS logs for full integration tests on lxd-container to ensure we didn't break anything outside of this minor changeset.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 22.1-14-g2e17a0d6-0ubuntu1~18.04.3

---------------
cloud-init (22.1-14-g2e17a0d6-0ubuntu1~18.04.3) bionic; urgency=medium

  * d/p/cpick-eee60329-Fix-cloud-init-status-wait-when-no-datasource-found:
    cherry-pick eee60329: Fix cloud-init status --wait when no datasource
    found (#1349) (LP: #1966085)

 -- James Falcon <email address hidden> Thu, 24 Mar 2022 10:49:10 -0500

Changed in cloud-init (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 22.1-14-g2e17a0d6-0ubuntu1~20.04.3

---------------
cloud-init (22.1-14-g2e17a0d6-0ubuntu1~20.04.3) focal; urgency=medium

  * d/p/cpick-eee60329-Fix-cloud-init-status-wait-when-no-datasource-found:
    cherry-pick eee60329: Fix cloud-init status --wait when no datasource
    found (#1349) (LP: #1966085)

 -- James Falcon <email address hidden> Thu, 24 Mar 2022 10:35:21 -0500

Changed in cloud-init (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 22.1-14-g2e17a0d6-0ubuntu1~21.10.3

---------------
cloud-init (22.1-14-g2e17a0d6-0ubuntu1~21.10.3) impish; urgency=medium

  * d/p/cpick-eee60329-Fix-cloud-init-status-wait-when-no-datasource-found:
    cherry-pick eee60329: Fix cloud-init status --wait when no datasource
    found (#1349) (LP: #1966085)

 -- James Falcon <email address hidden> Thu, 24 Mar 2022 10:32:19 -0500

Changed in cloud-init (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.