upgrade from Plucky to Questing cloud-init-base 25.3~2g890873f5-0ubuntu2 failed to install/upgrade on MAAS

[Patricia Domingues]

[ Impact ]

 * MAAS systems make use of a number of debconf settings which are read by cloud-init-base.postinst in order to create configuration files in /etc/cloud/cloud.cfg.d to instruct cloud-init how to obtain configuration information from the MAAS provisioning server. In the presence of multi-line settings, cloud-init-base.postinst fails to complete package upgrade resulting in invalid apt configuration and manual intervention of APT unattended upgrades on MAAS-provisioned machines and VMs. This postinst failure breaks automatic upgrades on MAAS systems.

[ Test Plan ]

 * Assert upgrade path to -proposed doesn't create unexpected /etc/cloud/cloud.cfg.d files due to cloud-init-base.postinst processing of debconf values on standard cloud images in non-MAAS environment

cat > setup_proposed.sh <<EOF
#!/usr/bin/bash
echo -e "Types: deb\nURIs: http://archive.ubuntu.com/ubuntu\nSuites: \$(lsb_release -sc)-proposed\nComponents: main universe restricted multiverse\nSigned-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg" | tee /etc/apt/sources.list.d/proposed.sources
apt-get update -q;
apt-get install -qy cloud-init;
EOF

lxc launch ubuntu-daily:questing test-2127022
lxc file push setup_proposed.sh test-2127022/

lxc exec test-2127022 bash

# Peek at existing debconf settings
debconf-show cloud-init > debconf.cloud-init.orig
debconf-show cloud-init-base >> debconf.cloud-init.orig

# Copy existing /etc/cloud/cloud.cfg.d to ensure no new files on upgrade
cp -r /etc/cloud/cloud.cfg.d .

bash /setup_proposed.sh
# Upgrade and ensure exit 0 and no errors
echo $?

debconf-show cloud-init > debconf.cloud-init.new
debconf-show cloud-init-base >> debconf.cloud-init.new

# expect no new config files created by postinst
diff -urN cloud.cfg.d /etc/cloud/cloud.cfg.d/

 * Assert multi-line debconf settings used by MAAS are properly processed and written to /etc/cloud/cloud.cfg.d by cloud-init-base.postinst

# provide multi-line value matching MAAS provisioning config debconf settings
cat > selections <<EOF
cloud-init cloud-init/local-cloud-config string manage_etc_hosts: true\nmanual_cache_clean: true
EOF

lxc launch ubuntu-daily:questing test-2127022

lxc file push selections test-2127022/
lxc exec test-2127022 -- debconf-set-selections /selections

# install from -proposed
lxc file push setup_proposed.sh test-21272022/

lxc exec test-2127022 bash

# Peek at existing debconf settings
debconf-show cloud-init > debconf.cloud-init.orig
debconf-show cloud-init-base >> debconf.cloud-init.orig

# Copy existing /etc/cloud/cloud.cfg.d to ensure no new files on upgrade
cp -r /etc/cloud/cloud.cfg.d .

bash /setup_proposed.sh
# Upgrade and ensure exit 0 and no errors
echo $?

debconf-show cloud-init > debconf.cloud-init.new
debconf-show cloud-init-base >> debconf.cloud-init.new

# expect 1 new config file created by postinst 90_dpkg_cloud_init_local.cfg
diff -urN cloud.cfg.d /etc/cloud/cloud.cfg.d/

[ Where problems could occur ]

 * Problems processing cloud-init or cloud-init-base debconf settings can show up across package upgrade path due to cloud-init-base.postinst use of db_get and db_set tooling to read and write debconfig configuration settings. Errors during package upgrade can result in non-zero exit codes from cloud-init-base.postinst which can result in errors affecting automatic upgrades which would force manual intervention to resolve or ignore such post install errors as seen in the original bug

 * Specific to MAAS debconf value processing, invalid handling of db_get results could result regressions by creating invalid cloud-config artifacts in /etc/cloud/cloud.cfg.d which would generate cloud-init schema warnings at boot resulting from exit 2 from such tools as `cloud-init status`.
This type of regression has been seen in Resolute series https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2131809

[ Other Info ]

===== Original description =====

Trying to upgrade a Power10 node from Ubuntu 25.04 Plucky to 25.10 Questing (do-release-upgrade -d) :

```
Found initrd image: /boot/initrd.img-6.14.0-33-generic
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Processing triggers for ca-certificates (20250419) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for initramfs-tools (0.150ubuntu3) ...
update-initramfs: Generating /boot/initrd.img-6.17.0-5-generic
Errors were encountered while processing:
 cloud-init-base
 cloud-init
needrestart is being skipped since dpkg has failed
packages have been installed but needrestart is suspended
Exception during pm.DoInstall(): E:Sub-process /usr/bin/dpkg returned an error code (1)

*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (345.0 KB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C):

```
It finishes the upgrade, but requires a manual reboot.
Then I can boot it into Questing.

[Patricia Domingues (patriciasd)]

adding this as there's a slightly similar issue with s390x https://bugs.launchpad.net/bugs/2126934

[Patricia Domingues (patriciasd)]

same also happens on arm64 server
```
Found initrd image: /boot/initrd.img-6.14.0-33-generic
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Adding boot menu entry for UEFI Firmware Settings ...
done
Processing triggers for ca-certificates (20250419) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for shim-signed (1.59+15.8-0ubuntu2) ...
Processing triggers for initramfs-tools (0.150ubuntu3) ...
update-initramfs: Generating /boot/initrd.img-6.17.0-5-generic
Errors were encountered while processing:
 cloud-init-base
 cloud-init
needrestart is being skipped since dpkg has failed
packages have been installed but needrestart is suspended
Exception during pm.DoInstall(): E:Sub-process /usr/bin/dpkg returned an error code (1)

*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (515.9 KB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C): K
Problem report file: /var/crash/cloud-init-base.0.crash

Could not install the upgrades

The upgrade has aborted. Your system could be in an unusable state. A
recovery will run now (dpkg --configure -a).

Setting up cloud-init-base (25.3~2g890873f5-0ubuntu2) ...
Migrating debconf value: cloud-init/local-cloud-config -> cloud-init-base/local-cloud-config
dpkg: error processing package cloud-init-base (--configure):
 installed cloud-init-base package post-installation script subprocess returned error exit status 128
dpkg: dependency problems prevent configuration of cloud-init:
 cloud-init depends on cloud-init-base (= 25.3~2g890873f5-0ubuntu2); however:
  Package cloud-init-base is not configured yet.

dpkg: error processing package cloud-init (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 cloud-init-base
 cloud-init

Upgrade complete

The upgrade has completed but there were errors during the upgrade
process.

To continue please press [ENTER]

To continue please press [ENTER]
=== Command detached from window (Tue Oct 7 16:04:08 2025) ===
```
after a manual reboot, can boot `Welcome to Ubuntu 25.10 (GNU/Linux 6.17.0-5-generic aarch64)`

[Patricia Domingues (patriciasd)]

ppc64el:
root@P10d-lpar11:/home/ubuntu# debconf-show cloud-init
* cloud-init/datasources: MAAS
* cloud-init/maas-metadata-url:
* cloud-init/local-cloud-config: manage_etc_hosts: true
manual_cache_clean: true
reporting:
  maas:
    consumer_key: ***
    endpoint: ***
    token_key: ***
    token_secret: ***
    type: webhook

* cloud-init/maas-metadata-credentials: oauth_consumer_key=***&oauth_token_key=***&oauth_token_secret=***

root@P10d-lpar11:/home/ubuntu# debconf-show cloud-init-base
* cloud-init-base/datasources: MAAS
```

arm64:

```
root@pompy:/home/ubuntu# debconf-show cloud-init
* cloud-init/local-cloud-config: manage_etc_hosts: true
manual_cache_clean: true
reporting:
  maas:
    consumer_key: ***
    endpoint: ***
    token_key: ***
    token_secret: ***
    type: webhook

* cloud-init/datasources: MAAS
* cloud-init/maas-metadata-credentials: oauth_consumer_key=***&oauth_token_key=***&oauth_token_secret=***
* cloud-init/maas-metadata-url:

root@pompy:/home/ubuntu# debconf-show cloud-init-base
* cloud-init-base/datasources: MAAS
```

[Patricia Domingues (patriciasd)]

The reported issue is contained to MAAS install, at least on Power. I was able to upgrade Plucky > Questing without the mentioned issue - when installing the system with the .ISO (https://cdimage.ubuntu.com/releases/25.04/release/ubuntu-25.04-live-server-ppc64el.iso).

packages have been installed but needrestart is suspended packages have been installed but needrestart is suspended ^M^[[0m^[[?25h^[[KAll snaps up to date.
System upgrade is complete.

Restart required

To finish the upgrade, a restart is required.
If you select 'y' the system will be restarted.

Continue [yN] y

[Chad Smith (chad.smith)]

Thank you @patriciasd for the bug.

I can confirm failure path reproducer in cloud-init-base.postinst when upgrading systems which have set a debconfig multi-line value for cloud-init/local-cloud-config.

In practice, this is only set in certain MAAS environments MAAS server < 3.5 and upgrades from cloud-init in Plucky to Questing.

I'll have a PPA up to allow testing prior to SRU to questing.

We also will add separate integration-test coverage in cloud-init's existing testing to preseed known MAAS debconf settings in order to better exercise this postinst path in the future.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 25.4~1gcb12e00e-0ubuntu1

---------------
cloud-init (25.4~1gcb12e00e-0ubuntu1) resolute; urgency=medium

  * d/cloud-init-base.config: update migrate_debconf_settings to
    set an empty cloud-init/datasources value once it migrates the value to
    cloud-init-base/datasources. (LP: #2127022)
  * d/cloud-init-base.postinst: fix release upgrade affecting the set of
    multi-line debconf value cloud-init/local-cloud-config used by MAAS.
    (LP: #2127022)

 -- Chad Smith <email address hidden> Fri, 10 Oct 2025 16:58:05 -0600

[Mitchell Augustin (mitchellaugustin)]

> In practice, this is only set in certain MAAS environments MAAS server < 3.5 and upgrades from cloud-init in Plucky to Questing

FYI, I think I just experienced this as well 2x on machines deployed in our MAAS running v3.5.8 (https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2130379)

[Timo Aaltonen (tjaalton)]

please add the required sru information in the description

[Timo Aaltonen (tjaalton)]

ping? it should not have been sponsored without the required SRU info

[Chad Smith (chad.smith)]

Thank you tjaalton. I have updated to SRU template to represent upgrade path with and without MAAS-specific debconf settings to establish correct behavior in both common case versus MAAS-specific case.

[Nick Rosbrook (enr0n)]

Hello Patricia, or anyone else affected,

Accepted cloud-init into questing-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/25.3~2g890873f5-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-questing to verification-done-questing. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-questing. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chad Smith (chad.smith)]

Questing Validation complete with success for both scenario 1 (with no MAAS debconf settings) and scenario 2 (with MAAS debconf setting)

Tracking SUCCESS comments on commandline for the above test procedure.

csmith@midtown:~/src/cloud-init$ lxc launch ubuntu-daily:questing test-2127022
Launching test-2127022
csmith@midtown:~/src/cloud-init$ lxc file push setup_proposed.sh test-2127022/
csmith@midtown:~/src/cloud-init$ lxc exec test-2127022 bash
root@test-2127022:~# debconf-show cloud-init > debconf.cloud-init.orig
debconf-show cloud-init-base >> debconf.cloud-init.orig
root@test-2127022:~# cat debconf.cloud-init.orig
* cloud-init/datasources: NoCloud, ConfigDrive, AltCloud, OVF, MAAS, Ec2, None
  cloud-init-base/datasources: NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, VMware, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, Oracle, Exoscale, RbxCloud, UpCloud, Vultr, LXD, NWCS, Akamai, WSL, CloudCIX, None
root@test-2127022:~# cp -r /etc/cloud/cloud.cfg.d .
root@test-2127022:~# bash /setup_proposed.sh
Types: deb
URIs: http://archive.ubuntu.com/ubuntu
Suites: questing-proposed
Components: main universe restricted multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
Get:1 http://security.ubuntu.com/ubuntu questing-security InRelease [136 kB]
Get:2 http://archive.ubuntu.com/ubuntu questing-proposed InRelease [275 kB]
Get:3 http://security.ubuntu.com/ubuntu questing-security/main amd64 Packages [106 kB]
Get:4 http://security.ubuntu.com/ubuntu questing-security/main Translation-en [29.2 kB]
Get:5 http://security.ubuntu.com/ubuntu questing-security/main amd64 Components [208 B]
Get:6 http://security.ubuntu.com/ubuntu questing-security/main amd64 c-n-f Metadata [3132 B]
Get:7 http://security.ubuntu.com/ubuntu questing-security/universe amd64 Packages [74.0 kB]
Get:8 http://security.ubuntu.com/ubuntu questing-security/universe Translation-en [22.9 kB]
Get:9 http://security.ubuntu.com/ubuntu questing-security/universe amd64 Components [6764 B]
Get:10 http://security.ubuntu.com/ubuntu questing-security/universe amd64 c-n-f Metadata [2344 B]
Get:11 http://security.ubuntu.com/ubuntu questing-security/restricted amd64 Packages [82.5 kB]
Get:12 http://security.ubuntu.com/ubuntu questing-security/restricted Translation-en [18.4 kB]
Get:13 http://security.ubuntu.com/ubuntu questing-security/restricted amd64 Components [208 B]
Get:14 http://security.ubuntu.com/ubuntu questing-security/multiverse amd64 Packages [1676 B]
Get:15 http://security.ubuntu.com/ubuntu questing-security/multiverse Translation-en [616 B]
Get:16 http://security.ubuntu.com/ubuntu questing-security/multiverse amd64 Components [212 B]
Get:17 http://security.ubuntu.com/ubuntu questing-security/multiverse amd64 c-n-f Metadata [224 B]
Hit:18 http://archive.ubuntu.com/ubuntu questing InRelease
Get:19 http://archive.ubuntu.com/ubuntu questing-updates InRelease [136 kB]
Get:20 http://archive.ubuntu.com/ubuntu questing-backports InRelease [133 kB]
Get:21 http://archive.ubuntu.com/ubuntu questing-proposed/main amd64 Packages [127 kB]
Get:22 http://archive.ubuntu.com/ubuntu questing-proposed/main...

[Chad Smith (chad.smith)]

Note this bugfix SRU blocks a more general cloud-init 25.3 SRU LP: #2131604 which will follow immediately after this bugfix only SRU publishes to questing.

[Nick Rosbrook (enr0n)]

Hello Patricia, or anyone else affected,

Accepted cloud-init into questing-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/25.3-0ubuntu1~25.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-questing to verification-done-questing. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-questing. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Patricia Domingues (patriciasd)]

Ok. installed

cloud-init_25.3-0ubuntu1~25.10.1_all.deb (2.1 KiB)

and

cloud-init-base_25.3-0ubuntu1~25.10.1_all.deb (610.4 KiB) from questing-proposed and then was successfully able to upgrade from P>Q, without the issue mentioned.

Foreign Packages Installed

The following unofficial packages are currently installed:

cloud-init...............Installed from: unknown origin
cloud-init-base..........Installed from: unknown origin

It is recommended to install supported versions from the Ubuntu
archive, and try the upgrade again.

Do you want to continue the upgrade anyways?

Continue [yN] y
...
System upgrade is complete.

Restart required

To finish the upgrade, a restart is required.
If you select 'y' the system will be restarted.

Continue [yN] y
...
ubuntu@P10rain-LPAR10:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 25.10
Release: 25.10
Codename: questing

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 25.3-0ubuntu1~25.10.1

---------------
cloud-init (25.3-0ubuntu1~25.10.1) questing; urgency=medium

  * d/p/deprecation-version-boundary.patch: Pin deprecation version to 25.3
  * Upstream snapshot based on 25.3. (LP: #2131604).
    List of changes from upstream can be found at
    https://raw.githubusercontent.com/canonical/cloud-init/25.3/ChangeLog

cloud-init (25.3~2g890873f5-0ubuntu3) questing; urgency=medium

  * d/cloud-init-base.config: update migrate_debconf_settings to
    set an empty cloud-init/datasources value once it migrates the value to
    cloud-init-base/datasources. (LP: #2127022)
  * d/cloud-init-base.postinst: fix release upgrade affecting the set of
    multi-line debconf value cloud-init/local-cloud-config used by MAAS.
    (LP: #2127022)

 -- Chad Smith <email address hidden> Thu, 11 Dec 2025 11:36:31 -0700

[Julian Andres Klode (juliank)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.