esm-cache.service throw apt.cache.FetchFailedException when connecting to https://esm.ubuntu.com fails
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
ubuntu-advantage-tools (Ubuntu) |
Fix Released
|
Critical
|
Unassigned | ||
Xenial |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Undecided
|
Unassigned | ||
Kinetic |
Fix Released
|
Undecided
|
Unassigned | ||
Lunar |
Fix Released
|
Critical
|
Unassigned |
Bug Description
[Impact]
Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service.
This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops.
The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless.
[Test Case]
On a machine with ubuntu-
make esm.ubuntu.com unreachable
then
```
apt update
systemctl --failed
ll /var/crash
```
You'll see that esm-cache.service failed and that it created a crash file in /var/crash.
To test the fix:
Install 27.13.3
```
apt update
systemctl --failed
ll /var/crash
```
esm-cache.service should succeed and there should be no new crash reports
We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-
GH PR that introduces the fix and automated test: https:/
[Regression Potential]
If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms.
[Discussion]
After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages.
This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway.
[Original Description]
Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https:/
$ journalctl -o cat -u esm-cache.service
Starting Update the local ESM caches...
Traceback (most recent call last):
File "/usr/lib/
main(cfg)
File "/usr/lib/
update_
File "/usr/lib/
cache.update()
File "/usr/lib/
raise FetchFailedExce
apt.cache.
esm-cache.service: Main process exited, code=exited, status=1/FAILURE
esm-cache.service: Failed with result 'exit-code'.
Failed to start Update the local ESM caches.
This also shows as a failed unit:
$ systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● esm-cache.service loaded failed failed Update the local ESM caches
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
1 loaded units listed.
Additional information:
In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https:/
$ apt-cache policy ubuntu-
ubuntu-
Installed: 27.13.2~22.04.1
Candidate: 27.13.2~22.04.1
Version table:
*** 27.13.2~22.04.1 500 (phased 0%)
500 https:/
100 /var/lib/
27.7~22.04.1 500
500 https:/
$ lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04
$ uname -a
Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Related branches
- Andreas Hasenack: Approve
- Robie Basak: Approve
- Canonical Server Reporter: Pending requested
-
Diff: 203 lines (+112/-7)6 files modifieddebian/changelog (+14/-0)
debian/ubuntu-advantage-tools.preinst (+31/-5)
features/steps/network.py (+23/-0)
features/unattached_commands.feature (+36/-0)
uaclient/apt.py (+7/-1)
uaclient/version.py (+1/-1)
Changed in ubuntu-advantage-tools (Ubuntu): | |
importance: | Undecided → Critical |
assignee: | nobody → Renan Rodrigo (renanrodrigo) |
status: | Confirmed → In Progress |
tags: | added: regression-update |
description: | updated |
description: | updated |
Changed in ubuntu-advantage-tools (Ubuntu): | |
assignee: | Renan Rodrigo (renanrodrigo) → nobody |
description: | updated |
description: | updated |
description: | updated |
This might be a bit user visible if the crash reporter causes a popup to show on desktops:
# ll /var/crash/ _usr_lib_ ubuntu- advantage_ esm_cache. py.0.crash _usr_lib_ ubuntu- advantage_ esm_cache. py.0.crash
-rw-r----- 1 root root 23576 Jan 27 12:48 /var/crash/