cloud-init is broken by cloud-init-base package on do-release-upgrade to plucky

[Benjamin Drung]

[ Impact ]

Cloud-init is currently broken on `do-release-upgrade` to plucky. In
plucky, cloud-init's packaging has been broken out into multiple binary
packages:
- cloud-init-base contains almost all of what used to be cloud-init
  minus some dependencies that aren't needed everywhere
- Additional cloud-specific packages will be made that contain
  cloud-init-base + the dependencies needed for that cloud
- cloud-init now depends on cloud-init-base + all extra cloud-specific
  dependencies.

This allows the cloud-init package to be essentially identical to what
the cloud-init package contained before with the advantage of being
able to install only `cloud-init-base` on systems that don't need
everything required by `cloud-init`'s full dependencies.

When making this change, the following was initially added to
`cloud-init-base` package on plucky:

Breaks: cloud-init (<= 24.4~3+really24.3.1-0ubuntu4)
Replaces: cloud-init (<= 24.4~3+really24.3.1-0ubuntu4)

However, this doesn't work. Since cloud-init get's SRUed in full, the
version numbers will always need to increase. Furthermore, this means
`cloud-init-base` can potentially break and replace the version of
cloud-init on plucky being upgraded.

Instead, we can fix this problem by adding the following to currently
supported released series:
Provides: cloud-init-base

while adding the following to the current devel release:
Replaces: cloud-init (<< [version]), cloud-init-base

[ Test Plan ]

- Launch an instance and upgrade to the proposed version. Verify that after upgrade cloud-init works without error and that no apt errors occurred.
- Launch an instance and do-release-upgrade to the next release. Verify that after upgrade, cloud-init works without and that no apt errors occurred.
- After the do-release-upgrade, list all cloud-init* packages installed and ensure expected ownership of files.

[ Where problems could occur ]

There could still be hidden errors that won't be unmasked until upgrading to a later version of cloud-init or upgrading from a newer version than specified in the "Replaces" line. However, since "Replaces:" is version gated, we would still have the opportunity to fix any new problem at that time.

[ Other Info ]

Adding `Replaces: cloud-init` to devel was the originally proposed solution, but Julian Klode suggested the current approach.

[ Original Description]

This crash happened on the upgrade from oracular to plucky.

ProblemType: Package
DistroRelease: Ubuntu 25.04
Package: cloud-init-base 24.4-0ubuntu1
ProcVersionSignature: Ubuntu 6.11.0-13.14-generic 6.11.0
Uname: Linux 6.11.0-13-generic x86_64
ApportVersion: 2.31.0-0ubuntu1
Architecture: amd64
CasperMD5CheckResult: pass
Date: Sat Dec 21 14:20:25 2024
ErrorMessage: Versuch, »/etc/cloud/cloud.cfg« zu überschreiben, welches auch in Paket cloud-init 24.4-0ubuntu1~24.10.1 ist
InstallationDate: Installed on 2024-10-15 (67 days ago)
InstallationMedia: Ubuntu-Cinnamon 24.04.1 LTS "Noble Numbat" - Release amd64 (20240827)
PackageArchitecture: all
Python3Details: /usr/bin/python3.12, Python 3.12.8, python3-minimal, 3.12.7-1
PythonDetails: N/A
RelatedPackageVersions:
 dpkg 1.22.11ubuntu3
 apt 2.9.16
SourcePackage: cloud-init
Title: package cloud-init-base 24.4-0ubuntu1 failed to install/upgrade: Versuch, »/etc/cloud/cloud.cfg« zu überschreiben, welches auch in Paket cloud-init 24.4-0ubuntu1~24.10.1 ist
UpgradeStatus: Upgraded to plucky on 2024-12-21 (0 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Paride Legovini (paride)]

Links to follow progress:

- https://github.com/canonical/cloud-init/pull/6137
- https://github.com/canonical/cloud-init/pull/6139

Thanks James, Julian!

[James Falcon (falcojr)]

Plucky, oracular, and noble packages have been uploaded.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 25.1.1-0ubuntu2

---------------
cloud-init (25.1.1-0ubuntu2) plucky; urgency=medium

  * d/control: Fix how cloud-init-base overwrites cloud-init files.
    (LP: #2092333)

 -- James Falcon <email address hidden> Fri, 28 Mar 2025 16:43:14 -0500

[Andreas Hasenack (ahasenack)]

Please include the following in the test plan:

- After the do-release-upgrade to plucky, list all cloud-init* packages installed, and compare to what is expected. From what I understood, two should be installed in the end: cloud-init, and cloud-init-base.

[Andreas Hasenack (ahasenack)]

Hello Benjamin, or anyone else affected,

Accepted cloud-init into oracular-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.4.1-0ubuntu0~24.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-oracular to verification-done-oracular. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-oracular. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Benjamin, or anyone else affected,

Accepted cloud-init into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/24.4.1-0ubuntu0~24.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Falcon (falcojr)]

Oracular upgrading cloud-init 24.4.1-0ubuntu0~24.10.2 to 24.4.1-0ubuntu0~24.10.3:
- No errors on upgrade
- No errors on reboot
- `dpkg -l | grep cloud-init` shows no cloud-init-base, though `apt install cloud-init-base` reports `Note, selecting 'cloud-init' instead of 'cloud-init-base'. cloud-init is already the newest version (24.4.1-0ubuntu0~24.10.3).`

Oracular `do-release-upgrade -d` from 24.4.1-0ubuntu0~24.10.3 to 25.1.1-0ubuntu2:
- `dpkg -l` shows cloud-init and cloud-init-base both installed.
- `systemctl status cloud-init*` shows success from all expected services/target/socket
- `dpkg -L cloud-init-base` shows ownership of almost all files
- `dpkg -L cloud-init` shows the few expected files.
- inspecting `/var/log/cloud-init.log`, cloud-init ran after reboot following do-release-upgrade with no errors.

Noble upgrading cloud-init 24.4.1-0ubuntu0~24.04.2 to 24.4.1-0ubuntu0~24.04.3:
- No errors on upgrade
- No errors on reboot
- `dpkg -l | grep cloud-init` shows no cloud-init-base, though `apt install cloud-init-base` reports `Note, selecting 'cloud-init' instead of 'cloud-init-base'. cloud-init is already the newest version (24.4.1-0ubuntu0~24.10.3).`

`do-release-upgrade` is currently not possible from Noble as no newer LTS exists to upgrade to.

[Timo Aaltonen (tjaalton)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.4.1-0ubuntu0~24.10.3

---------------
cloud-init (24.4.1-0ubuntu0~24.10.3) oracular; urgency=medium

  * d/control: Fix how cloud-init-base overwrites cloud-init files.
    (LP: #2092333)

 -- James Falcon <email address hidden> Wed, 02 Apr 2025 10:02:44 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 24.4.1-0ubuntu0~24.04.3

---------------
cloud-init (24.4.1-0ubuntu0~24.04.3) noble; urgency=medium

  * d/control: Fix how cloud-init-base overwrites cloud-init files.
    (LP: #2092333)

 -- James Falcon <email address hidden> Wed, 02 Apr 2025 10:09:15 -0500