Show Extended Security Maintenence status

[Robert Ancell]

[Impact]
There is not currently a graphical method of determining if a system is subscribed to [Extended Security Maintenance](https://ubuntu.com/security/esm) updates. This is resolved by adding some [new UI](https://wiki.ubuntu.com/SoftwareUpdates#Extended_Security_Maintenance) to the software properties application.

[Test Case]
1. Install latest version of Ubuntu advantage:
$ sudo add-apt-repository ppa:ua-client/stable
$ sudo apt update
$ sudo apt upgrade
2. Open Software Properties
3. Go to Updates tab.

Expected result:
Information is shown that indicates if this system is using Extended Security Maintenance updates, when updates will supported until, and a link to upgrade to ESM.

Observed result:
No ESM information currently shown.

[Where problems could occur]
- Software properties could hit a bug getting a response from the ua app. The current code carefully checks if and what is returned, falling back to a safe default behavior.
- Launching software properties could trigger a bug in the ua app.
- Software properties could show incorrect information, causing confusion for the user. The solution uses information from distro-info and the ua app which means software-properties contains no data about ESM, and instead relies on these apps that can be updated if things change.

[Robert Ancell (robert-ancell)]

I've made a PPA to test this: https://launchpad.net/~robert-ancell/+archive/ubuntu/software-properties

[Robert Ancell (robert-ancell)]

Code is in:
https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+ref/esm
https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+ref/esm-xenial

[Robert Ancell (robert-ancell)]

Merge request for master:
https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+merge/400153

Other branches for SRUing:
https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+ref/esm-focal
https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+ref/esm-bionic

[Chad Smith (chad.smith)]

Thanks @robert-ancell for setting up the PPA.

I tested on Xenial Ubuntu Desktop LiveCD and would like to report Failure on this implementation.

Your use of subprocess is passing capture_output on xenial and that is raising an error in python before calling ua client.

On a system attached to esm-apps:

python3 -c 'import gi; gi.require_version("Gtk", "3.0"); from softwareproperties.gtk.utils import get_esm_apps_status; print(get_esm_apps_status())'
Failed to call ubuntu advantage client:
__init__() got an unexpected keyword argument 'capture_output'
(False, False)

Note that run(capture_output) param is not available on python3.5 (it was introduced only in 3.7).

Also just dropping that capture_output param you've got some other logic relying on result.stdout which is None in this case.

while you iterate on this work, you could substitute esm-infra in your get_esm_apps_status function as it'll behave the same way as app, and all contracts are entitled to esm-infra at the moment so it will be easier to test enabled, available etc.

If you'd like review on the iterations of these SRU branches related to this just assign lamoura or chad.smith too and we can take a look.

[Chad Smith (chad.smith)]

Another thing to note/expect: ua client in release in xenial and bionic at the moment is a completed incompatible commandine (which we are planning on SRUing prior to Xenial ESM, so `ua status --format=json` will not work on something < version 20. I'm fairly certain you shouldn't care to support this old version of ua-tools, but you might want to just do a pre-flight check on Xenial and bionic 'ua version' >= 20 and return (False, False) in that case notiing that those environments should update their versionn of ubuntu-advantage-tools.

[Robert Ancell (robert-ancell)]

This change is based on the expectation that UA tools will be the version that is about to be SRUd. The old version gracefully fails with `ua status --format=json`, so should already effectively return (False, False). I didn't do an explicit check on the version as it's non-trivial to do and I'd be more worried about introducing a bug in the version number parsing.

[Robert Ancell (robert-ancell)]

I've updated the xenial branch and version in the PPA with the fixes requested, thanks!

[Robert Ancell (robert-ancell)]

Bionic and focal branches/PPA also updated.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.99.13

---------------
software-properties (0.99.13) impish; urgency=medium

  * Catch exception if UA status file is not written

 -- Robert Ancell <email address hidden> Fri, 11 Jun 2021 17:16:09 +1200

[Timo Aaltonen (tjaalton)]

Hello Robert, or anyone else affected,

Accepted software-properties into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Robert, or anyone else affected,

Accepted software-properties into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.9.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Robert, or anyone else affected,

Accepted software-properties into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.96.24.32.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[corrado venturini (corradoventu)]

Installed from proposed on Focal: works fine!

[corrado venturini (corradoventu)]

on Focal software-properties-gtk gives some error messages and seems very slow to start
corrado@corrado-n5-focal:~$ time software-properties-gtk
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'

real 0m14,651s
user 0m3,872s
sys 0m0,692s
corrado@corrado-n5-focal:~$

[corrado venturini (corradoventu)]

Also on Hirsute Hippo is ok but gives error messages:
corrado@corrado-n7-hippo:~$ time software-properties-gtk
No ua status file written:
[Errno 2] No such file or directory: '/var/lib/ubuntu-advantage/status.json'

real 0m10,718s
user 0m2,522s
sys 0m0,497s
corrado@corrado-n7-hippo:~$

[Robert Ancell (robert-ancell)]

The warnings about the status.json file are due to the ubuntu-advantage client on these older systems not yet having the feature to write this file. When updated, this should work - that's correct, right @chad.smith?

[Chad Smith (chad.smith)]

@robert-ancell and @corradoventu those errors are due to running software-properties-gtk as non-root user on a system that is not attached to any ubuntu-advantage services.

 The /var/lib/ubuntu-advantage/status.json file is only created by a root user via an `sudo ua status` or `sudo ua attach <token>` call. The absence of the file means that the system is not attached to any valid Ubuntu Advantage services, and the logic in softwareproperties/gtk/util.py will still correctly return an empty representation for ubuntu-advantage services as an empty dict {} and the UX dialogs properly direct folks to how to activate ESM in this case.

I mistakenly thought software-properties-gtk is invoked as root user in order to install/update packages. Given that it is invoked as non-root user we can do one of two things:
   1. keep the current implementation which would print messages about missing status.json files knowing that the UX dialogs still behave correctly.
OR
   2. drop the subp call to 'ua' 'status' from utils.py:get_ua_status() because they will never emit the /var/lib/ubuntu-advantage/status.json artifact and also drop the printed messages about No ua status file written as that really only means you are non-root and on an unattached machine.

[Chad Smith (chad.smith)]

Also per #2 above, if we were to drop the `ua status` call from utils.py:get_ua_status() that would also mitigate the perceived "time cost" corradoventu mentioned because `ua status` does reach out to the internet to contracts.canonical.com to check service availability for your running distro/kernel/platform. If we don't make that `ua status` commandline call, and rely on presence/absence of /var/lib/ubuntu-advantage/status.json then we can avoid the round trip to an external URL to fill in these service details.

[Chad Smith (chad.smith)]

Ultimately if we decide to go with #2 here's a patch that I think would limit both the time cost of a remote uRL query and the noisy logging:

--- /usr/lib/python3/dist-packages/softwareproperties/gtk/utils.py 2021-06-10 23:20:22.000000000 -0600
+++ /usr/lib/python3/dist-packages/softwareproperties/gtk/utils.py.new 2021-07-14 15:32:54.559693094 -0600
@@ -26,6 +26,7 @@
 gi.require_version("Gtk", "3.0")
 from gi.repository import Gio, Gtk
 import json
+import os
 import subprocess

 import logging
@@ -76,23 +77,12 @@
 def get_ua_status():
     """Return a dict of all UA status information or empty dict on error."""
     # status.json will exist on any attached system or any unattached system
- # which has already run `ua status`. Calling ua status directly on
- # network disconnected machines will raise a TimeoutException trying to
- # access contracts.canonical.com/v1/resources.
- try:
- # Success writes UA_STATUS_JSON
- result = subprocess.run(['ua', 'status', '--format=json'], stdout=subprocess.PIPE)
- except Exception as e:
- print("Failed to call ubuntu advantage client:\n%s" % e)
+ # which has already run `sudo ua status`.
+ if not os.path.exists(UA_STATUS_JSON):
         return {}
- if result.returncode != 0:
- print("Ubuntu advantage client returned code %d" % result.returncode)
- return {}
-
     try:
         status_file = open(UA_STATUS_JSON, "r")
     except Exception as e:
- print("No ua status file written:\n%s" % e)
         return {}

     with status_file as stream:

[Chad Smith (chad.smith)]

miniamlly as in the patch above, I think we probably need a pre-flight check of os.path.exists( UA_STATUS_JSON) so we don't emit those Errno messages due to failed file opens

[Robert Ancell (robert-ancell)]

Confirmed here that /var/lib/ubuntu-advantage/status.json is only updated when running 'ua status' as root.

No desktop software is running as root, in the few cases that root privileges are required a small service is run as root and that is communicated with via D-Bus.

The warning doesn't show for the majority of users, since they aren't going to be running from the command line so I would leave it for now, and remove in the next revision of this code.

I think there is a case where this won't work - if you have a fresh machine that you haven't done a 'ua attach' on then software-properties is going to not offer ESM as it's not going to think it's available.

In https://code.launchpad.net/~robert-ancell/software-properties/+git/software-properties/+merge/400153 that 'ua status' can timeout and return empty values, which is why we are accessing /var/lib/ubuntu-advantage/status.json. Is there a reason why the ua client doesn't do this itself internally, i.e. it gives you fresh json or returns the cached copy if it can't do the update?

[Chad Smith (chad.smith)]

>> Is there a reason why the ua client doesn't do this itself internally, i.e. it gives you fresh json or returns the cached copy if it can't do the update?

Currently the only "time" during which ubuntu-advantage-tools (UA client) could perform the logic to exec 'sudo ua status' is during postinst at package install upgrade time. UA client can not perform that status setup during package install for the same timeout reasons you mentioned. It is a potentially long operation. If run during a package install it could delay or prevent other package upgrades if it hits a timeout and/or error. That said, we are working on instrumenting a systemd timer service that can run config change/setup operations out of band of typical APT package installs. So, this case could be handled by that timer-based service. This SRU does represent.

This changeset as-is does represent an improvement to existing cases because the Software properties UX demonstrates awareness of ESM (and contract expiry information) properly for attached machines, or unattached machines where `sudo ua status` has been run.

Subsequent releases of software-properties will now be able to take advantage of reading "ua status --format=json" output directly as non-root (due to a bug-fix released in 27.2.1). The non-root user will not have to rely anymore on that json file if absent as the --format=json output will render the same JSON dict that would be present on disk for the root user. This is a fix that is under verification in the -proposed queue now for ua-tools and was not available at the time of this original software-properties PR. I'm ok with this approach as-is knowing that we can significantly improve the behavior once ubuntu-advantage-tools 27.2.1 is released to X B F and H

[Chad Smith (chad.smith)]

Thank you for this submission. I was awaiting feedback on this bug to determine if you'd like to solve the outstanding issue that Extend... links aren't presented on the software-properties-gtk UI for unattached machines (and/or) dropping the error messages from the console. But, as you mentioned error console messaging isn't presented to the UI so this is a non-issue that we can remedy in future uploads.

That said, I also discovered that changes in ubuntu-advantage-tools 27.2 which have already SRU'd into xenial, bionic, focal and render this current logic broken.

I've filed LP: #1939732 to capture upcoming features needed in the UI to better inform about Extending your support if desired on Bionic/Focal etc.

LP 1939732 could be fixed by this diff:

--- /usr/lib/python3/dist-packages/softwareproperties/gtk/utils.py.old 2021-08-12 13:56:59.695413561 -0600
+++ /usr/lib/python3/dist-packages/softwareproperties/gtk/utils.py 2021-08-12 14:12:28.597659801 -0600
@@ -26,6 +26,7 @@
 gi.require_version("Gtk", "3.0")
 from gi.repository import Gio, Gtk
 import json
+import os
 import subprocess

 import logging
@@ -79,24 +80,20 @@
     # which has already run `ua status`. Calling ua status directly on
     # network disconnected machines will raise a TimeoutException trying to
     # access contracts.canonical.com/v1/resources.
- try:
- # Success writes UA_STATUS_JSON
- result = subprocess.run(['ua', 'status', '--format=json'], stdout=subprocess.PIPE)
- except Exception as e:
- print("Failed to call ubuntu advantage client:\n%s" % e)
- return {}
- if result.returncode != 0:
- print("Ubuntu advantage client returned code %d" % result.returncode)
- return {}
-
- try:
- status_file = open(UA_STATUS_JSON, "r")
- except Exception as e:
- print("No ua status file written:\n%s" % e)
- return {}
-
- with status_file as stream:
- status_json = stream.read()
+ if os.path.exists(UA_STATUS_JSON):
+ with open(UA_STATUS_JSON) as stream:
+ status_json = stream.read()
+ else:
+ try:
+ # Success writes UA_STATUS_JSON
+ result = subprocess.run(['ua', 'status', '--format=json'], stdout=subprocess.PIPE)
+ except Exception as e:
+ print("Failed to call ubuntu advantage client:\n%s" % e)
+ return {}
+ if result.returncode != 0:
+ print("Ubuntu advantage client returned code %d" % result.returncode)
+ return {}
+ status_json = result.stdout
     try:
         status = json.loads(status_json)
     except json.JSONDecodeError as e:

 The UX "updates" tab no longer can correctly determine that the machine is attached because a new field "available" has been published in status.json.

Current utils.py:get_ua_service_status relies on a now incorrect check:

            if service.get("available"): # then we are not attached
                if service["available"] == "yes":
                     available = True
                     service_status = "disabled" # Disabled since unattached

Since ua status output on attach machines now provides "available": "yes" in ve...

[Robert Ancell (robert-ancell)]

@chad.smith - can you make merge requests for the required changes to the ubuntu/hirsute, ubuntu/focal etc branches (https://git.launchpad.net/ubuntu/+source/software-properties)? I can then review those and do the uploads as required. If you need any UI changes I can make those, but I think at this point it's just how to interact with the ua client correctly which is something you have a lot more information on than I do.

[Chad Smith (chad.smith)]

@robert, looks like we should reject this queued upload and put up a separate set of uploads. Recommending reject here and we can work this & next week to get you MRs to support current best-practices for interacting with ua status.json.

[Robert Ancell (robert-ancell)]

@chad.smith - agreed, these changes as currently proposed will need to be updated.

[Corey Bryant (corey.bryant)]

@robert, @chad.smith - would you like me to revert the commit from the ubuntu/focal branch for this https://git.launchpad.net/software-properties ?

[Chad Smith (chad.smith)]

I'm hesitant for your to revert on the focal branches as I don't have context on if there is tooling that consumes those branches directly in the project. I don't think it's too much for me to get you and/or Robert a patch for this now. Today is good for me and I'll push a couple PRs for you and Robert to review.

[Brian Murray (brian-murray)]

Hello Robert, or anyone else affected,

Accepted software-properties into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Robert, or anyone else affected,

Accepted software-properties into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/software-properties/0.99.9.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chad Smith (chad.smith)]

# Focal desktop results: SUCCESS

1. # install proposed

$ cat > setup_proposed.sh <<EOF
#!/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q
apt-get install -qy software-properties-gtk software-properties-common python3-software-properties
EOF
$ IP=`uvt-kvm ip ubuntu20.04`
$ scp setup_proposed.sh csmith@$IP:.

csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ lsb_release -sc
focal
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ apt policy software-properties-gtk
software-properties-gtk:
  Installed: 0.99.9.8
  Candidate: 0.99.9.8
  Version table:
 *** 0.99.9.8 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main i386 Packages
        100 /var/lib/dpkg/status
     0.98.9.5 500
        500 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu focal-updates/main i386 Packages
     0.98.9.2 500
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages
     0.98.9 500
        500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu focal/main i386 Packages
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ [ -f /var/lib/ubuntu-advantage-tools/status.json ] && echo "status.json PRESENT" || echo "status.json ABSENT"
status.json ABSENT
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ software-properties-gtk
# no errors
3. Updates tab in "Software & Updates" dialog
Basic Security Maintenance Extended... # Link goes to https://ubuntu.com/security/esm
Active until 04/23/2025

4. Cancel button
# no errors on software-properties-gtk console
5. Attach and validate
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ sudo ua attach <REDACTED>
[sudo] password for csmith:
Enabling default service esm-apps
Updating package lists
UA Apps: ESM enabled
Enabling default service esm-infra
Updating package lists
UA Infra: ESM enabled
Enabling default service livepatch
Canonical livepatch enabled.
This machine is now attached to 'server-team-ua-client-ci-uaa'

SERVICE ENTITLED STATUS DESCRIPTION
cis yes disabled Center for Internet Security Audit Tools
esm-apps yes enabled UA Apps: Extended Security Maintenance (ESM)
esm-infra yes enabled UA Infra: Extended Security Maintenance (ESM)
fips yes disabled NIST-certified core packages
fips-updates yes disabled NIST-certified core packages with priority security updates
livepatch yes enabled Canonical Livepatch service

NOTICES
Operation in progress: ua attach

Enable services with: ua enable <service>

                Account: UA Client Test
           Subscription: server-team-ua-client-ci-uaa
            Valid until: 9999-12-31 00:00:00+00:00
Technical support level: essential
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ software-properties-gtk
# no token expiry error messages
6. Click Updates tab and validate:
Extended Security Maintenance
Active until 12/31/1999

[Chad Smith (chad.smith)]

# Impish desktop results: SUCCESS non-LTS has neither ESM-Infra nor ESM-Apps support or availability.
# so 1. it will not expose "Extend..." links to provide ESM information in Updates tab
# and 2. When attached no ESM information will be represented in Updates tab

1. # install proposed

$ cat > setup_proposed.sh <<EOF
#!/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q
apt-get install -qy software-properties-gtk software-properties-common python3-software-properties
EOF'
$ IP=`uvt-kvm ip ubuntuimpish`
$ scp setup_proposed.sh csmith@$IP:.
$ ssh csmith@$IP
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ lsb_release -sc
impish
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ apt policy software-properties-gtk
software-properties-gtk:
  Installed: 0.99.13.1
  Candidate: 0.99.13.1
  Version table:
 *** 0.99.13.1 500
        500 http://archive.ubuntu.com/ubuntu impish-proposed/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu impish-proposed/main i386 Packages
        100 /var/lib/dpkg/status
     0.99.13 500
        500 http://us.archive.ubuntu.com/ubuntu impish/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu impish/main i386 Packages
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ [ -f /var/lib/ubuntu-advantage/status.json ] && echo "status.json PRESENT" || echo "status.json ABSENT"
status.json ABSENT

csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ sofware-properties-gtk
# Success NO error messages

3. Updates tab in "Software & Updates" dialog
# Success no "Extend..." link
Basic Security Maintenance
Active until 07/14/2022

4. Close button
# no errors on software-properties-gtk console

5. Attach and validate
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ sudo ua attach <REDACTED>

This machine is now attached to 'server-team-ua-client-ci-uaa'

SERVICE ENTITLED STATUS DESCRIPTION
cis yes n/a Center for Internet Security Audit Tools
esm-infra yes n/a UA Infra: Extended Security Maintenance (ESM)
fips yes n/a NIST-certified core packages
fips-updates yes n/a NIST-certified core packages with priority security updates
livepatch yes n/a Canonical Livepatch service

NOTICES
Operation in progress: ua attach

Enable services with: ua enable <service>

                Account: UA Client Test
           Subscription: server-team-ua-client-ci-uaa
            Valid until: 9999-12-31 00:00:00+00:00
Technical support level: essential

csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ software-properties-gtk
# no token expiry error messages
6. Click Updates tab and validate:
# Success Basic Security Maintenance because no ESM available or active on non-LTS
Basic Security Maintenance
Active until 07/14/2022

[Chad Smith (chad.smith)]

@brian-murray the verification-done-impish tag may be unnecessary here by me. My above verification on impish may have just been supplemental as 0.99.13 already had released which provided ESM awareness on Impish. That said, I revalidated this because the current 0.99.13.1 also touched that same logic and I wanted to confirm no regression.

[Chad Smith (chad.smith)]

# Hirsute desktop results: SUCCESS non-LTS has neither ESM-Infra nor ESM-Apps support or availability.
# so 1. it will not expose "Extend..." links to provide ESM information in Updates tab
# and 2. When attached no ESM information will be represented in Updates tab

1. # install proposed

$ cat > setup_proposed.sh <<EOF
#!/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q
apt-get install -qy software-properties-gtk software-properties-common python3-software-properties
EOF'
$ IP=`uvt-kvm ip ubuntuhirsute`
$ scp setup_proposed.sh csmith@$IP:.
$ ssh csmith@$IP
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ lsb_release -sc
hirsute
csmith@csmith-Standard-PC-i440FX-PIIX-1996:~$ sudo bash ./setup_proposed.sh
[sudo] password for csmith:
deb http://archive.ubuntu.com/ubuntu hirsute-proposed main
Hit:1 http://us.archive.ubuntu.com/ubuntu hirsute InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu hirsute-updates InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu hirsute-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu hirsute-security InRelease
Get:5 http://archive.ubuntu.com/ubuntu hirsute-proposed InRelease [269 kB]
Get:6 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 Packages [61.4 kB]
Get:7 http://archive.ubuntu.com/ubuntu hirsute-proposed/main i386 Packages [20.1 kB]
Get:8 http://archive.ubuntu.com/ubuntu hirsute-proposed/main Translation-en [18.2 kB]
Get:9 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 DEP-11 Metadata [6,228 B]
Get:10 http://archive.ubuntu.com/ubuntu hirsute-proposed/main DEP-11 48x48 Icons [6,469 B]
Get:11 http://archive.ubuntu.com/ubuntu hirsute-proposed/main DEP-11 64x64 Icons [10.0 kB]
Get:12 http://archive.ubuntu.com/ubuntu hirsute-proposed/main DEP-11 64x64@2 Icons [29 B]
Get:13 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 c-n-f Metadata [1,712 B]
Fetched 393 kB in 2s (230 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be upgraded:
  python3-software-properties software-properties-common
  software-properties-gtk
3 upgraded, 0 newly installed, 0 to remove and 235 not upgraded.
Need to get 112 kB of archives.
After this operation, 15.4 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 software-properties-common all 0.99.10.2 [13.9 kB]
Get:2 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 software-properties-gtk all 0.99.10.2 [65.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu hirsute-proposed/main amd64 python3-software-properties all 0.99.10.2 [32.1 kB]
Fetched 112 kB in 1s (101 kB/s)
(Reading database ... 168161 files and directories currently installed.)
Preparing to unpack .../software-properties-common_0.99.10.2_all.deb ...
Unpacking software-properties-common (0.99.10.2) over (0.99.10) ...
Preparing to unpack .../software-properties-gtk_0.99.10.2_all.deb ...
Unpacking software-properties-gtk (0.99.10.2) over (0.99.10) ...
Preparing to unpack .../python3-software-properties_0.99.10.2_all.deb...

[Chad Smith (chad.smith)]

I think there was an omission on the latest bionic upload it doesn't contain the fixes that were added for focal, hirsute and impish.

I've put up a PR that cherry-picks those fixes into the bionic branch that would need review and upload before I can validate and set verification-done-bionic on this bug.

bionic upstream PR.
https://code.launchpad.net/~chad.smith/software-properties/+git/software-properties/+merge/411393

[Robert Ancell (robert-ancell)]

This has been uploaded as 0.96.24.32.18. SRU team please reject 0.96.24.32.17 in proposed as no need for that SRU to delay this one.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.99.10.2

---------------
software-properties (0.99.10.2) hirsute; urgency=medium

  * utils: prefer /var/lib/ubuntu-advantage/status.json over ua status
    - Handle absent /var/lib/ubuntu-advantage/status.json for non-root
      users (LP: #1939732)
    - print unexcepted errors and if _schema_version not equal to 0.1

software-properties (0.99.10.1) hirsute; urgency=medium

  * Show ESM support status (LP: #1920836)

 -- Chad Smith <email address hidden> Fri, 29 Oct 2021 16:35:58 -0600

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for software-properties has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.99.9.8

---------------
software-properties (0.99.9.8) focal; urgency=medium

  [ Corey Bryant ]
  * cloudarchive: Enable support for the Yoga Ubuntu Cloud Archive on
    20.04 (LP: #1948806).

  [ Chad Smith ]
  * utils: prefer /var/lib/ubuntu-advantage/status.json over ua status
    - Handle absent /var/lib/ubuntu-advantage/status.json for non-root
      users (LP: #1939732)
    - print unexcepted errors and if _schema_version not equal to 0.1

software-properties (0.99.9.7) focal; urgency=medium

  * Show Ubuntu Pro banner on Livepatch page (LP: #1934439)

software-properties (0.99.9.6) focal; urgency=medium

  * Show ESM support status (LP: #1920836)

 -- Corey Bryant <email address hidden> Tue, 26 Oct 2021 08:27:56 -0400

[Chad Smith (chad.smith)]

# Success testing on Bionic Desktop installer
1. Validate current behavior from bionic-updates
csmith@downtown:~$ IP=`uvt-kvm ip ubuntu18.04`
csmith@downtown:~$ ssh csmith@$IP
Welcome to Ubuntu 18.04.6 LTS (GNU/Linux 5.4.0-84-generic x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

8 updates can be applied immediately.
8 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

New release '20.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Your Hardware Enablement Stack (HWE) is supported until April 2023.
Last login: Thu Nov 18 14:29:55 2021 from 192.168.122.1
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ cat > setup_proposed.sh <<EOF
> #!/bin/bash
> mirror=http://archive.ubuntu.com/ubuntu
> echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
> apt-get update -q
> apt-get install -qy software-properties-gtk software-properties-common python3-software-properties
> EOF
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ lsb_release -sc
bionic
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ apt policy software-properties-gtk
software-properties-gtk:
  Installed: 0.96.24.32.14
  Candidate: 0.96.24.32.14
  Version table:
 *** 0.96.24.32.14 500
        500 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages
        100 /var/lib/dpkg/status
     0.96.24.32.1 500
        500 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu bionic/main i386 Packages
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ [ -f /var/lib/ubuntu-advantage-tools/status.json ] && echo "status.json PRESENT" || echo "status.json ABSENT"
status.json ABSENT
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ software-properties-gtk
# no error messages
# Updates tab on Bionic does not currently report any ESM, support expiry info

2. Install -proposed software-properties, click Updates tab show the correct expiry for Bionic on unattached
csmith@csmith-Standard-PC-Q35-ICH9-2009:~$ sudo bash ./setup_proposed.sh
deb http://archive.ubuntu.com/ubuntu bionic-proposed main
Hit:1 http://us.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://us.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-proposed InRelease [242 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,378 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,303 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages [114 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic-proposed/main i386 Packages [80.6 kB]
Get:10 http://archive.ub...

[Launchpad Janitor (janitor)]

This bug was fixed in the package software-properties - 0.96.24.32.18

---------------
software-properties (0.96.24.32.18) bionic; urgency=medium

  [ Chad Smith ]
  * utils: prefer /var/lib/ubuntu-advantage/status.json over ua status
    - Handle absent /var/lib/ubuntu-advantage/status.json for non-root
      users (LP: #1939732)
    - print unexpected errors and if _schema_version not equal to 0.1

software-properties (0.96.24.32.17) bionic; urgency=medium

  * Show Ubuntu Pro banner on Livepatch page (LP: #1934439)

 -- Robert Ancell <email address hidden> Fri, 05 Nov 2021 16:44:17 +1300