[Impact]
The main feature in this release is bringing esm-apps to GA. Beta flags and text are removed, and the service is available for all LTS.
There is also an important bug fix, where we no longer change the system apt configuration to get esm package information (LP: #1990378).
Other changes include:
* Enabling autocomplete for the 'pro' command (GH: #2280)
* Do not show invalid subcommands in autocomplete (GH: #2279)
* Support attaching when snapd is held (LP: #1997514)
* Do not fail if the apt-hook executable is not present (LP: #1994480)
* Redesign output of security-status to properly show support (LP: #2002407)
See the changelog entry below for a full list of changes and bugs.
[Test Case]
The following development and SRU process was followed:
https://wiki.ubuntu.com/UbuntuAdvantageToolsUpdates
The ubuntu-advantage-tools team will be in charge of attaching the artifacts and
console output of the appropriate run to the bug. ubuntu-advantage-tools team
members will not mark ‘verification-done’ until this has happened.
Additionally, to test the APT ESM setup, we will perform the following steps:
1) Launch a LXD machine and install version 27.13.1 of ubuntu-advantage-tools
2) Install the bpfcc-tools package on the host machine
3) Run the command:
sudo execsnoop-bpfcc -u LXD_USER_ID -T
Where LXD_USER_ID is the user id that LXD uses to run (Usually it is 1000000)
4) On the LXD container, run `apt update` and verify that only one `update-success-stamp` is created on the logs of `execsnoop-bpfcc` (which indicates that apt update ran only once)
* Automated Test Results
<TODO>
Attach or link the following automated integration test runs for ubuntu-advantage-tools on each affected LTS release
</TODO>
[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.
Removing the unauthenticated esm-repos from Xenial machines in postinst may remove the esm sources.list files even if we didn't create them. However, the chance for a user adding exactly this file is really low, and if the service is enabled, postinst keeps it there. Enabling the service would overwrite any file with the same name anyway.
If the local esm caches are not updated properly (when the service runs, as part of the pre-update hook), the user will not see any available esm updates if the service is disabled. This will not fail, it will just not show the lists until the update succeeds. We have integration tests to make sure it runs as fast and reliable as possible.
Now that apt news are server-based, a network fail may keep outdated messages in the apt output, or miss new messages. This is mitigated by using a local cache for the messages and, similar to the issue above, will fix itself as soon as the server can be reached again, and does not pose any error or fail.
Magic attach command line is only tested against a staging environment. We consider this enough for the release given:
- clear expectation that the production environment will work exactly the same, and
- a well tested flow using the API
The service is expected to be up by Jan 26th.
[Discussion]
The change in the esm package reports involves apt configuration, requiring a pre-SRU review. This review happened in the PR itself, by Julian Klode and Robie Basak:
https://github.com/canonical/ubuntu-advantage-client/pull/2342
Additionally, we identified that we need a more isolated APT ESM setup (see comment #1). By isolated, we mean that it should only have access to the necessary system APT configuration to work (For example, identify that a proxy is set in the system APT and use it too). This was achieved in this PR:
https://github.com/canonical/ubuntu-advantage-client/pull/2393
Which will be release in version 27.13.1
Finally, esm-apps was going to be GA last November, but the release date was postponed to January 26th. This SRU needs to land on -updates exactly on this date.
[Changelog]
* d/bash-completion:
- enable autocomplete for the 'pro' command (GH: #2280)
* d/control:
- update the package description
* d/postinst:
- remove unauthenticated esm repos from Xenial systems (LP: #1990378)
* New upstream release 27.13 (LP: #2003018)
- apt:
+ remove logic which added repositories and pinned them to 'never' to
enable access to esm package lists
+ add functionality to create and update a local apt esm cache with
the lists for esm-infra and esm-apps
- apt-hook: update the cpp hook to use the local esm apt cache
- apt-news:
+ fetch and display APT News in apt upgrade
+ show contract expiration notices in the apt news output
- attach: support attaching without being able to install snapd
(LP: #1997514)
- cli:
+ do not show invalid subcommands in autocomplete (GH: #2279)
+ add support for attaching through the web portal, without a token
- config: add apt_news_url option
- docs: reorganize documentation and correct information
- esm-apps: release the service as GA
- jobs:
+ remove the update_status job
+ remove unused job which checks for the system EOL
- messaging: do not fail if the apt-hook executable is not present
(LP: #1994480)
- motd: announce esm-apps as GA
- security-status:
+ use the local esm cache to report updates when the services are
disabled
+ redesign output to properly show support (LP: #2002407)
- services: add new service to update the local esm caches
- ros: release the service as GA
- bug fixes:
+ report reboot_required even if 'livepatch status' fails
+ do not create unexpected environment variables when the autocomplete
script runs
+ contract requests do not cause 'pro status' to fail
+ remove auto-attach motd message if any failure happens
+ log when 'cloud-id' fails
+ always honor the metering job timer config
+ write files atomically
In the SRU review it was noticed that the apt environment used to update the ESM cache is not really isolated from the system. For example, we noticed that the APT::Update:
APT::Update:
"[ ! -e /run/systemd/system ] || systemctl start --no-block apt-news.service esm-cache.service || true";
};
https:/
This happens because esm-cache.service, in the end, calls apt update again (via the python library). We just don't get a nasty loop here because systemd won't start a second copy of esm-cache.service.
This lack of isolation is a concern. All hooks from the system apt (defined in /etc/apt) will be called by the esm-only apt, maybe even in parallel, depending on timing. There are hooks to update stamp files, and motd. There are many unknowns here.
We discussed this at length with the Pro team, and will take the following approach:
- the Pro team work on isolating the esm-cache apt instance, and pull in only very specific configs from the system apt (/etc/apt), like proxy settings, and other settings they identify as being needed
- I will accept u-a-t into proposed, so that the other aspects of this SRU can be tested in parallel, but with the condition that: a) the Pro team will come up with a new set of fixes on top for the "shared config" issue identified above, and upload a new version to proposed; b) this package I'm accepting into proposed today MUST NOT be released. I will add a block-proposed tag to this bug to that effect
- when the improved package is ready to be uploaded, it shall be accepted in to proposed on top of today's package, and a new round of testing will be done. Initially only on top of the new changes, but if possible, the whole test suite.