[ ubuntu-advantage-tools SRU info ]
[ Impact ]
If ubuntu-advantage-tools (u-a-t) is not already installed, but cloud-init user-data includes "ubuntu_advantage" configuration data, then cloud-init will try to install u-a-t. On install, u-a-t needs to start ubuntu-advantage.service. ubuntu-advantage.service is ordered After=cloud-config.service. This results in deadlock, preventing apt from ever finishing. The apt lock is held forever and cloud-init never continues.
The fix is to break the deadlock by removing the "After=cloud-config.service" ordering dependency from ubuntu-advantage.service. ubuntu-advantage.service should still wait until cloud-config.service is done activating to do any work though, so that waiting is now implemented in the python script that is executed by the service.
[ Test Plan ]
```
# Launch a lxd container with ubuntu_advantage cloudinit user-data:
lxc launch ubuntu-daily:bionic test --config "user.user-data=#cloud-config\nubuntu_advantage: {}"
# Enter the container
lxc shell test
# Remove u-a-t
apt remove ubuntu-advantage-tools
# tell cloud init to clean and reboot
cloud-init clean --logs --reboot
# enter the container again
lxc shell test
# wait for cloud-init to finish
cloud-init status --wait
# if the bug is happening, that command will never finish
```
This test has been added as a regression test to u-a-t and will be run as part of the validation for LP: #2048921 (see: https://github.com/canonical/ubuntu-pro-client/blob/release-31/features/install_uninstall.feature#L116)
[ Where problems could occur ]
This does not remove the ordering dependency, but just changes how the ordering dependency between ubuntu-advantage.service and cloud-config.service is implemented. This difference in implementation could cause different unexpected ordering problems to occur in the future.
The waiting function in python was designed such that it will eventually continue if cloud-config.service takes longer than 20 minutes. If cloud-config.service takes 20 minutes doing an Ubuntu Pro related operation (normal operations take less than 10 seconds at most), then ubuntu-advantage.service could continue and potentially error when it tries to do something in conflict with the ongoing operation. Alternatively, if the wait was implemented incorrectly, then ubuntu-advantage.service could stall forever unnecessarily if cloud-config.service doesn't finish.
[Original Description]
During install of Ubuntu Server 22.04.3 you are asked if you want to attach to Ubuntu Pro. If you do then following a reboot you cannot run apt. cloud-init runs and installs ubuntu-advantage-tools which for some reason never releases the lock file.
$ sudo apt dist-upgrade
Waiting for cache lock: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 994 (apt-get)
If you reboot at this point then you receive:
$ sudo apt dist-upgrade
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: ubuntu-advantage-tools 30~22.04
ProcVersionSignature: Ubuntu 6.5.0-14.14~22.04.1-generic 6.5.3
Uname: Linux 6.5.0-14-generic x86_64
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Sun Jan 21 23:11:59 2024
InstallationDate: Installed on 2024-01-21 (0 days ago)
InstallationMedia: Ubuntu-Server 22.04.3 LTS "Jammy Jellyfish" - Release amd64 (20230810)
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: ubuntu-advantage-tools
UpgradeStatus: No upgrade log present (probably fresh install)
cloud-id.txt: none
livepatch-status.txt-error: Invalid command specified '/snap/bin/canonical-livepatch status'.
pro-journal.txt: -- No entries --
uaclient.conf:
contract_url: https://contracts.canonical.com
log_level: debug
Hi Ken,
Thanks for reporting this. When looking at the ubuntu-
This might be an APT related problem. However, I am adding subiquity to this issue, since they are the ones that own the installer. They can assess the status of this issue and what we can do about it