Activity log for bug #2004130

Date Who What changed Old value New value Message
2023-01-30 00:41:41 Simon Déziel bug added bug
2023-01-30 00:46:17 Robie Basak bug added subscriber Robie Basak
2023-01-30 09:31:27 Launchpad Janitor ubuntu-advantage-tools (Ubuntu): status New Confirmed
2023-01-30 11:45:48 Renan Rodrigo ubuntu-advantage-tools (Ubuntu): importance Undecided Critical
2023-01-30 11:46:05 Renan Rodrigo ubuntu-advantage-tools (Ubuntu): assignee Renan Rodrigo (renanrodrigo)
2023-01-30 11:48:16 Renan Rodrigo ubuntu-advantage-tools (Ubuntu): status Confirmed In Progress
2023-01-30 12:09:30 Robie Basak tags regression-update
2023-01-30 15:18:32 Grant Orndorff description Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last): File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module> main(cfg) File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main update_esm_caches(cfg) File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches cache.update() File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools: Installed: 27.13.2~22.04.1 Candidate: 27.13.2~22.04.1 Version table: *** 27.13.2~22.04.1 500 (phased 0%) 500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages 100 /var/lib/dpkg/status 27.7~22.04.1 500 500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. TODO add link to source and command used to run automated test [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
2023-01-30 17:04:02 Andreas Hasenack bug added subscriber Andreas Hasenack
2023-01-30 17:04:44 Chris Johnston bug added subscriber Chris Johnston
2023-01-30 18:28:11 Grant Orndorff description [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. TODO add link to source and command used to run automated test [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
2023-01-30 19:34:28 Launchpad Janitor merge proposal linked https://code.launchpad.net/~orndorffgrant/ubuntu/+source/ubuntu-advantage-tools/+git/ubuntu-advantage-tools/+merge/436562
2023-01-30 20:58:37 Renan Rodrigo ubuntu-advantage-tools (Ubuntu): assignee Renan Rodrigo (renanrodrigo)
2023-01-31 14:10:56 Grant Orndorff description [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 As part of this SRU we'll also be testing the continuation of the temporary fix for #2003977. This will consist of ensuring the following upgrade paths do not cause conffile prompts: 27.11.3 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.11.3 with pro config set apt_news=false -> 27.13.3 27.11.3 with pro config set apt_news=true -> 27.13.3 27.12 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 with pro config set apt_news=false -> 27.13.3 27.12 with pro config set apt_news=true -> 27.13.3 27.13.1 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 with pro config set apt_news=false -> 27.13.3 27.13.1 with pro config set apt_news=true -> 27.13.3 27.13.2 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 with pro config set apt_news=false -> 27.13.3 27.13.2 with pro config set apt_news=true -> 27.13.3 [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
2023-01-31 14:14:26 Grant Orndorff description [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 As part of this SRU we'll also be testing the continuation of the temporary fix for #2003977. This will consist of ensuring the following upgrade paths do not cause conffile prompts: 27.11.3 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.11.3 with pro config set apt_news=false -> 27.13.3 27.11.3 with pro config set apt_news=true -> 27.13.3 27.12 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 with pro config set apt_news=false -> 27.13.3 27.12 with pro config set apt_news=true -> 27.13.3 27.13.1 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 with pro config set apt_news=false -> 27.13.3 27.13.1 with pro config set apt_news=true -> 27.13.3 27.13.2 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 with pro config set apt_news=false -> 27.13.3 27.13.2 with pro config set apt_news=true -> 27.13.3 [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 As part of this SRU we'll also be testing the continuation of the temporary fix for #2003977. This will consist of ensuring the following upgrade paths do not cause conffile prompts: 27.11.3 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.11.3 with pro config set apt_news=false -> 27.13.3 27.11.3 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 with pro config set apt_news=false -> 27.13.3 27.12 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 with pro config set apt_news=false -> 27.13.3 27.13.1 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 with pro config set apt_news=false -> 27.13.3 27.13.2 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
2023-01-31 14:43:59 Grant Orndorff description [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 As part of this SRU we'll also be testing the continuation of the temporary fix for #2003977. This will consist of ensuring the following upgrade paths do not cause conffile prompts: 27.11.3 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.11.3 with pro config set apt_news=false -> 27.13.3 27.11.3 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.12 with pro config set apt_news=false -> 27.13.3 27.12 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.1 with pro config set apt_news=false -> 27.13.3 27.13.1 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK 27.13.2 with pro config set apt_news=false -> 27.13.3 27.13.2 with pro config set apt_news=true -> 27.13.3 and debsums -e ubuntu-advantage-tools is OK [Regression Potential] We haven't done the proper fix for #2003977 yet, so if we didn't properly update the preinst hack to avoid prompts, that bug may reoccur. As part of this release we are ensuring the checksums in preinst are accurate and will still properly guard against this bug for upgrades from all applicable versions: 27.11.3, 27.12, 27.13.1, 27.13.2. Testing these upgrade paths will be a part of the verification for this SRU. If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux [Impact] Users who have systems that cannot reach esm.ubuntu.com for any reason will get crash reports and degraded systemd status from a failed esm-cache.service. This could result in alerts from monitoring setups on servers as well as crash report pop-ups on desktops. The service itself is non-critical so its okay for esm.ubuntu.com to be unreachable, but the alerts and pop-ups are concerning nonetheless. [Test Case] On a machine with ubuntu-advantage-tools 27.13.1 or 27.13.2 make esm.ubuntu.com unreachable then ``` apt update systemctl --failed ll /var/crash ``` You'll see that esm-cache.service failed and that it created a crash file in /var/crash. To test the fix: Install 27.13.3 ``` apt update systemctl --failed ll /var/crash ``` esm-cache.service should succeed and there should be no new crash reports We are creating an automated test for this scenario, which will run on all future SRUs of ubuntu-advantage-tools. This test will be used for verification of this SRU. GH PR that introduces the fix and automated test: https://github.com/canonical/ubuntu-advantage-client/pull/2404 [Regression Potential] If we made a mistake in editing esm-cache.service, we could catch this error but introduce a new one in the same service, which would result in the same symptoms. [Discussion] After this change, failures in esm-cache.service will not be as visible so machines may not receive messages about potential esm updates and they won't receive any noticeable indication that they are missing out on these messages. This is deemed acceptable since the messages are not critical to a functioning system and if esm.ubuntu.com is blocked, then the system won't be able to download the updates themselves anyway. [Original Description] Since u-a-t version 27.13.1~22.04.1, if something prevents successful connection to https://esm.ubuntu.com, the esm-cache.service will throw an exception: $ journalctl -o cat -u esm-cache.service Starting Update the local ESM caches... Traceback (most recent call last):   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 22, in <module>     main(cfg)   File "/usr/lib/ubuntu-advantage/esm_cache.py", line 11, in main     update_esm_caches(cfg)   File "/usr/lib/python3/dist-packages/uaclient/apt.py", line 636, in update_esm_caches     cache.update()   File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update     raise FetchFailedException() apt.cache.FetchFailedException esm-cache.service: Main process exited, code=exited, status=1/FAILURE esm-cache.service: Failed with result 'exit-code'. Failed to start Update the local ESM caches. This also shows as a failed unit: $ systemctl --failed   UNIT LOAD ACTIVE SUB DESCRIPTION ● esm-cache.service loaded failed failed Update the local ESM caches LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. Additional information: In my case, the communication failure is because I use a HTTPS proxy that doesn't allow outbound connections to https://esm.ubuntu.com. $ apt-cache policy ubuntu-advantage-tools ubuntu-advantage-tools:   Installed: 27.13.2~22.04.1   Candidate: 27.13.2~22.04.1   Version table:  *** 27.13.2~22.04.1 500 (phased 0%)         500 https://apt.sdeziel.info/archive/ubuntu jammy-updates/main amd64 Packages         100 /var/lib/dpkg/status      27.7~22.04.1 500         500 https://apt.sdeziel.info/archive/ubuntu jammy/main amd64 Packages $ lsb_release -rd Description: Ubuntu 22.04.1 LTS Release: 22.04 $ uname -a Linux sdeziel-lemur 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Bionic
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Bionic)
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Kinetic
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Kinetic)
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Jammy
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Jammy)
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Focal
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Focal)
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Xenial
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Xenial)
2023-01-31 15:04:09 Andreas Hasenack nominated for series Ubuntu Lunar
2023-01-31 15:04:09 Andreas Hasenack bug task added ubuntu-advantage-tools (Ubuntu Lunar)
2023-01-31 17:26:23 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Xenial): status New Confirmed
2023-01-31 17:26:23 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Bionic): status New Confirmed
2023-01-31 17:26:23 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Focal): status New Confirmed
2023-01-31 17:26:23 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Jammy): status New Confirmed
2023-01-31 17:26:23 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Kinetic): status New Confirmed
2023-01-31 20:54:44 Richard Mansfield bug added subscriber Richard Mansfield
2023-01-31 23:53:38 Robie Basak ubuntu-advantage-tools (Ubuntu Kinetic): status Confirmed Fix Committed
2023-01-31 23:53:39 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2023-01-31 23:53:41 Robie Basak bug added subscriber SRU Verification
2023-01-31 23:53:44 Robie Basak tags regression-update regression-update verification-needed verification-needed-kinetic
2023-01-31 23:54:14 Robie Basak ubuntu-advantage-tools (Ubuntu Jammy): status Confirmed Fix Committed
2023-01-31 23:54:17 Robie Basak tags regression-update verification-needed verification-needed-kinetic regression-update verification-needed verification-needed-jammy verification-needed-kinetic
2023-01-31 23:54:36 Robie Basak ubuntu-advantage-tools (Ubuntu Focal): status Confirmed Fix Committed
2023-01-31 23:54:40 Robie Basak tags regression-update verification-needed verification-needed-jammy verification-needed-kinetic regression-update verification-needed verification-needed-focal verification-needed-jammy verification-needed-kinetic
2023-01-31 23:54:56 Robie Basak ubuntu-advantage-tools (Ubuntu Bionic): status Confirmed Fix Committed
2023-01-31 23:55:00 Robie Basak tags regression-update verification-needed verification-needed-focal verification-needed-jammy verification-needed-kinetic regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic
2023-01-31 23:55:22 Robie Basak ubuntu-advantage-tools (Ubuntu Xenial): status Confirmed Fix Committed
2023-01-31 23:55:26 Robie Basak tags regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic verification-needed-xenial
2023-02-01 01:26:42 Simon Déziel tags regression-update verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy verification-needed-kinetic verification-needed-xenial regression-update verification-done-jammy verification-needed verification-needed-bionic verification-needed-focal verification-needed-kinetic verification-needed-xenial
2023-02-01 17:46:15 Grant Orndorff attachment added verification-2004130.tar.gz https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2004130/+attachment/5644302/+files/verification-2004130.tar.gz
2023-02-01 17:46:36 Grant Orndorff tags regression-update verification-done-jammy verification-needed verification-needed-bionic verification-needed-focal verification-needed-kinetic verification-needed-xenial regression-update verification-done verification-done-bionic verification-done-focal verification-done-jammy verification-done-kinetic verification-done-xenial
2023-02-01 23:16:15 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Lunar): status In Progress Fix Released
2023-02-02 12:20:20 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Kinetic): status Fix Committed Fix Released
2023-02-02 12:20:26 Andreas Hasenack removed subscriber Ubuntu Stable Release Updates Team
2023-02-02 12:20:50 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Jammy): status Fix Committed Fix Released
2023-02-02 12:21:17 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Focal): status Fix Committed Fix Released
2023-02-02 12:21:38 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Bionic): status Fix Committed Fix Released
2023-02-02 12:21:56 Launchpad Janitor ubuntu-advantage-tools (Ubuntu Xenial): status Fix Committed Fix Released