NoCloud SMBIOS seed broken in 23.4

[James Falcon]

[ Impact ]

In cloud-init 23.4, specifying a datasource via the SMBIOS is broken. This is due to a commit that fixed another bug of cloud-init assuming a datasource list consisting of one datasource and 'None' doesn't need to check for the existence of that datasource. The fix here is to ensure we're always checking for the SMBIOS information, whereas in the past we didn't need to because of the automatic datasource assumption.

[ Test Plan ]

An integration test has been added upstream as part of https://github.com/canonical/cloud-init/pull/4949 . This test fails if run against 23.4. If it passes, it demonstrates that the fix works. The test specifies an http datasource seed via SMBIOS and ensures that the userdata contained within that seed ran correctly. It can be run manually using:
tox -e integration-tests -- tests/integration_tests/datasources/test_nocloud.py::test_smbios_seed_network

[ Where problems could occur ]

Since this code sits at the interaction of datasources discovered and found via ds-identify and python, along with bits of fallback code to dmidecode, we could have missed an interaction here that could possibly mis-identify the NoCloud datasource in other ways.

[ Other Info ]

Upstream bug: https://github.com/canonical/cloud-init/issues/4945
Upstream dup: https://github.com/canonical/cloud-init/issues/4951

Documentation of specifying NoCloud datasource via SMBIOS :
https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html#method-2-local-filesystem-kernel-commandline-or-smbios

Upstream PR with change that broke the cloud-init behavior:
https://github.com/canonical/cloud-init/pull/4426

Upstream PR fixing the behavior:
https://github.com/canonical/cloud-init/pull/4949

[Chad Smith (chad.smith)]

Upstream PR landed fixing this issue:
https://github.com/canonical/cloud-init/commit/66b5ce9d5f94c0c6625972fdfdca3796d365b069

upstream commit with integration test for ongoing validaiton of this behavior:
https://github.com/canonical/cloud-init/commit/4b694a45db2f9043700e6a619bb943086be2f0c9

[Chad Smith (chad.smith)]

Uploads queued to focal jammy and mantic -proposed for review.

New integration tests cover this test case:

CLOUD_INIT_PLATFORM=lxd_vm tox -e integration-tests -- tests/integration_tests/datasources/test_nocloud.py::test_smbios_seed_network tests/integration_tests/datasources/test_nocloud.py::test_smbios_seed_local

[Chris Halse Rogers (raof)]

Hello James, or anyone else affected,

Accepted cloud-init into mantic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.4.4-0ubuntu0~23.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-mantic to verification-done-mantic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-mantic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chris Halse Rogers (raof)]

Hello James, or anyone else affected,

Accepted cloud-init into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.4.4-0ubuntu0~22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chris Halse Rogers (raof)]

Hello James, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/23.4.4-0ubuntu0~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.1~7g54599148-0ubuntu1

---------------
cloud-init (24.1~7g54599148-0ubuntu1) noble; urgency=medium

  * d/cloud-init.templates: enable WSL datasource by default.
  * Restore upstream ChangeLog without any merges.
    We have been merging an upstream ChangeLog spelling difference
    downstream for no good reason.
  * Upstream snapshot based on upstream/main at 54599148.
    - Bugs fixed in this snapshot: (LP: #2055081, #2055219)

 -- James Falcon <email address hidden> Tue, 27 Feb 2024 17:42:55 -0600

[James Falcon (falcojr)]

This includes the 23.4.4 integration test run results. All failures are expected and explained in a file named "failures.txt".

The test described in this bug's test plan is included in these runs. It can be found by searching for "tests/integration_tests/datasources/test_nocloud.py::TestSmbios::test_smbios_seed_network". It PASSED on all series.

[Andreas Hasenack (ahasenack)]

Given the urgency on this hotfix, I'm releasing it early.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 23.4.4-0ubuntu0~23.10.1

---------------
cloud-init (23.4.4-0ubuntu0~23.10.1) mantic; urgency=medium

  * Upstream snapshot based on 23.4.4. (LP: #2055081).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.4.4/ChangeLog

 -- Chad Smith <email address hidden> Tue, 27 Feb 2024 08:17:49 -0700

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 23.4.4-0ubuntu0~22.04.1

---------------
cloud-init (23.4.4-0ubuntu0~22.04.1) jammy; urgency=medium

  * Upstream snapshot based on 23.4.4. (LP: #2055081).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.4.4/ChangeLog

 -- Chad Smith <email address hidden> Tue, 27 Feb 2024 08:14:44 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 23.4.4-0ubuntu0~20.04.1

---------------
cloud-init (23.4.4-0ubuntu0~20.04.1) focal; urgency=medium

  * Upstream snapshot based on 23.4.4. (LP: #2055081).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/23.4.4/ChangeLog

 -- Chad Smith <email address hidden> Tue, 27 Feb 2024 08:13:27 -0700