esm security updates not reported by apt update-notifier

Bug #1881632 reported by Chad Smith on 2020-06-01
270
This bug affects 2 people
Affects Status Importance Assigned to Milestone
update-notifier (Ubuntu)
Undecided
Andreas Hasenack
Trusty
Undecided
Andreas Hasenack
Xenial
Undecided
Unassigned
Bionic
Undecided
Unassigned
Focal
Undecided
Unassigned

Bug Description

[Impact]
ESM-related Security pocket packages are not reported being classified as security due to a rename in the backend apt suites from esm-security -> esm-infra-security and esm-apps-security.

[Test Case]
* Launch a trusty/xenial/bionic/focal lxd from ua-client/proposed PPA.

* Run the script that displays the motd bit about available updates:
sudo /usr/lib/update-notifier/apt-check --human-readable

* The output should be something like this, signaling there are only ESM updates available:
"""
UA Infrastructure Extended Security Maintenance (ESM) is not enabled.

0 updates can be installed immediately.
0 of these updates are security updates.

Enable UA Infrastructure ESM to receive 88 additional security updates.
See https://ubuntu.com/advantage or run: sudo ua status
"""

* Obtain an UA token for free at https://ubuntu.com/advantage

* Run attach:
sudo ua attach <token-obtained-in-previous-step>

* Confirm that esm-infra was enabled:
sudo ua status

* Run this command again to display the motd banner output about available updates:
sudo /usr/lib/update-notifier/apt-check --human-readable

* You should get something like this without the fix for this bug:
"""
UA Infrastructure Extended Security Maintenance (ESM) is enabled.

89 updates can be installed immediately.
89 of these updates are provided through UA Infrastructure ESM.
0 of these updates are security updates.
To see these additional updates run: apt list --upgradable
"""

* In the output above, which is without the fix, note how none of the available updates are flagged as security

* With the updated update-notifier package, the security updates count correctly includes the ESM security updates:
"""
UA Infrastructure Extended Security Maintenance (ESM) is enabled.

88 updates can be installed immediately.
88 of these updates are provided through UA Infrastructure ESM.
85 of these updates are security updates.
To see these additional updates run: apt list --upgradable
"""

Test Script:
#!/bin/bash
#
# SRU Verification update-notifier + ubuntu=advantage-tools
# Test procedure:
# - launch container Trusty, Xenial or Bionic
# - Install ubuntu-advantage-tools from https://launchpad.net/~ua-client/+archive/ubuntu/proposed which supports esm on trusty, xenial, bionic, and focal
# - Attach container to UA subscription (which activates the ESM APT repos
# - run apt_check --human-readable to assert ESM pkg counts ARE NOT reported
# - Upgrade update-notifier to -proposed
# - re-run apt_check --human-readable to assert ESM pkg counts ARE reported

set -e
UA_TOKEN=$1
if [ -z "$1" ]; then
 echo "Usage: $0 <contractTOKEN>"
 exit 1
fi
# sources:
# ua.proposed:
# source: deb http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu \$RELEASE main
# keyid: 94E187AD53A59D1847E4880F8A295C4FB8B190B7

cat > test-uru.yaml <<EOF
#cloud-config
ssh_import_id: [chad.smith]
package_update: true
package_upgrade: true
apt:
  sources:
      ua.proposed: deb http://ppa.launchpad.net/ua-client/proposed/ubuntu \$RELEASE main
      keyid: 6E34E7116C0BC933
EOF

cat > setup_proposed.sh <<EOF
#/bin/bash
mirror=http://archive.ubuntu.com/ubuntu
echo deb \$mirror \$(lsb_release -sc)-proposed main | tee /etc/apt/sources.list.d/proposed.list
apt-get update -q
apt-get install -qy update-notifier
EOF

wait_for_boot() {
  local vm=$1 release=$2
  echo "--- Wait for cloud-init to finish"
  sleep 5
    lxc exec ${vm} -- cloud-init status --wait --long
}

for release in xenial bionic focal; do
  echo "--- BEGIN $release update-notifier testing"
  vm=test-sru-$release
  echo "--- Launch cloud-init with ppa:ua-client/proposed enabled"
  lxc launch ubuntu-daily:${release} ${vm} -c user.user-data="$(cat test-un.yaml)"
  wait_for_boot ${vm} ${release}
  echo "--- Attach Ubuntu-Advantage, enabling services"
  lxc exec ${vm} -- ua attach ${UA_TOKEN}
  echo "--- Check Original MOTD output from apt_check before upgrade"
  lxc exec ${vm} -- /usr/lib/update-notifier/apt-check --human-readable
  echo "--- Upgrade update-notifier from -proposed"
  lxc file push setup_proposed.sh ${vm}/
  lxc exec ${vm} -- bash /setup-proposed.sh | grep update-notifier
  echo "--- Check upadate-notifier(-proposed) MOTD output from apt_check after upgrade"
  lxc exec ${vm} -- /usr/lib/update-notifier/apt-check --human-readable
done

[Regression Potential]
The fix is replacing the old incorrect name (<distro>-security) of the ESM security pocket with the correct one (<distro>-infra-security). The old name came from the old ubuntu-advantage-tools bash client, version 10. If this name remains incorrect, the security update coming from ESM won't be counted, which is exactly this bug. So the regression potential in this one liner is that it remains uncounted.

[Other Info]
Instead of fixing the pocket's name, we could have *added* a new pocket with the current correct name, since the server part of ESM responds to both trusty-security and trusyt-infra-security (with origin UbuntuESM).

The reasons we didn't do that are:
- only the old bash client (version 10) used the old pocket name, and it's not available for trusty anymore (unless you go to https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+publishinghistory and fetch it)
- there was a concern about potentially counting updates twice, if both trusty-security and trusty-infra-security were enabled at the same time
- the upgrade from the bash client (v10) to the current client DOES NOT change the pocket name in the sources.list snippet for ESM, so in that brief moment after an upgrade and before a reattach, the count would be zero just like in this bug. HOWEVER, it's a known process that after upgrading from the bash client to the current one, the machine has to be attached again. See the last paragraph of the description in https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1832757, which is when the non-bash client was SRUed, reproduced below:
"""
On an upgrade, existing users of trusty esm are expected to run "sudo ua attach [<token>]", although not doing it won't disable their existing ESM access. The new ua tool just won't recognize esm as being active in its "ua status" output until the attach operation is complete. The same applies to livepatch, if it was enabled before.
"""
The process of attaching will rewrite the pocket name in the local sources.list file snippet from trusty-security to trusty-infra-security.

Finally, this update is for trusty only. Xenial doesn't have ESM yet, and updating update-notifier there would be an useless download for users, with a regression risk for no benefit.

[Original Description]

ESM-related Security pocket packages are not reported being classified as security due to a rename in the backend apt suites from esm-security -> esm-infra-security and esm-apps-security.

The customer issue reported catches the symptom well:

"""

I believe there's a problem with "apt_check.py" in the "update-notifier-common" package when using "ua". I have enabled "ua" via "ua attach" and yet "apt-check" shows updates, but does not specify they are security updates, even though they are:
mrussell@deputy:~$ /usr/lib/update-notifier/apt-check --human-readable
UA Infrastructure Extended Security Maintenance (ESM) is enabled.

8 updates can be installed immediately.
8 of these updates are provided through UA Infrastructure ESM.
0 of these updates are security updates.
To see these additional updates run: apt list --upgradable

Note, these are the packages:
mrussell@deputy:~$ apt list --upgradable
Listing... Done
apt/trusty-infra-security 1.0.1ubuntu2.24+esm1 amd64 [upgradable
from: 1.0.1ubuntu2.24]
apt-transport-https/trusty-infra-security 1.0.1ubuntu2.24+esm1 amd64 [upgradable from: 1.0.1ubuntu2.24]
apt-utils/trusty-infra-security 1.0.1ubuntu2.24+esm1 amd64 [upgradable from: 1.0.1ubuntu2.24]
libapt-inst1.5/trusty-infra-security 1.0.1ubuntu2.24+esm1 amd64 [upgradable from: 1.0.1ubuntu2.24]
libapt-pkg4.12/trusty-infra-security 1.0.1ubuntu2.24+esm1 amd64 [upgradable from: 1.0.1ubuntu2.24]
libjson-c2/trusty-infra-security 0.11-3ubuntu1.2+esm3 amd64 [upgradable from: 0.11-3ubuntu1.2+esm2]
libjson0/trusty-infra-security 0.11-3ubuntu1.2+esm3 amd64 [upgradable from: 0.11-3ubuntu1.2+esm2]

If I change "isSecurityUpgrade()" to also include this
value in "security_pockets": ("UbuntuESM", "%s-infra-security" % DISTRO),
then, the output is correct:
mrussell@deputy:~$ /usr/lib/update-notifier/apt-check --human-readable
UA Infrastructure Extended Security Maintenance (ESM) is enabled.

8 updates can be installed immediately.
8 of these updates are provided through UA Infrastructure ESM.
8 of these updates are security updates.
To see these additional updates run: apt list --upgradable
"""

Related branches

Chad Smith (chad.smith) wrote :

Patch suggestion to ensure both ESM Apps and ESM Infra security pockets are properly classified as security type updates

ESM infra pocket: esm-infra-security
ESM apps pocket: esm-apps-security

Chad Smith (chad.smith) on 2020-06-01
information type: Public → Private Security
Chad Smith (chad.smith) on 2020-06-01
information type: Private Security → Public Security
tags: added: patch
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in update-notifier (Ubuntu):
status: New → Confirmed
Dimitri John Ledkov (xnox) wrote :

I guess this needs to go all the way back to trusty, right?

tags: added: rls-gg-incoming
Mark Morlino (markmorlino) wrote :

yes, all the way back to trusty.

Mark Morlino (markmorlino) wrote :

I can upload the fix to trusty ESM. Should the other releases go to the -security or the -updates pocket?

Julian Andres Klode (juliank) wrote :

Patch looks good to me, does it need sponsoring?

We released other ESM enablement fixes via -updates, so this probably does not need pushing via security.

Mark Morlino (markmorlino) wrote :

yes, needs sponsoring.

And actually, it looks like there have also been a few other updates to update-notifier that went to -updates for trusty even after standard support ended. They appear to have been similarly dealing with how it reports ESM updates. Perhaps I was wrong in my previous comment and it makes more sense to publish this in -updates for trusty instead of trusty/esm?

Andreas Hasenack (ahasenack) wrote :

Note that this bit needs to be ESM_ORIGINS, not esm_origins:

@@ -58,7 +61,7 @@ def isSecurityUpgrade(ver):
 def isESMUpgrade(ver):
     " check if the given version is a security update (or masks one) "
     for (file, index) in ver.file_list:
- if file.origin == "UbuntuESM" and file.archive.startswith(DISTRO):
+ if file.origin in esm_origins and file.archive.startswith(DISTRO):
             return True
     return False

Andreas Hasenack (ahasenack) wrote :

There are other ESM changes done to the trusty package which have never been SRUed to the later ubuntu releases :/

Changed in update-notifier (Ubuntu):
assignee: nobody → Andreas Hasenack (ahasenack)
status: Confirmed → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.192.32

---------------
update-notifier (3.192.32) groovy; urgency=medium

  [ Chad Smith ]
  * data/apt_check.py: Update ESM security pockets names (LP: #1881632)
    - the UbuntuESM pocket was renamed from <distro>-security to
      <distro>-infra-security
    - new origin UbuntuESMApps, with a corresponding pocket of
      <distro>-apps-security

 -- Andreas Hasenack <email address hidden> Fri, 12 Jun 2020 11:21:25 -0300

Changed in update-notifier (Ubuntu):
status: In Progress → Fix Released
description: updated
description: updated
Changed in update-notifier (Ubuntu Trusty):
status: New → In Progress
assignee: nobody → Andreas Hasenack (ahasenack)
Andreas Hasenack (ahasenack) wrote :

By checking an update against ESM_ORIGINS, the current patch is aggregating esm security updates for both infra and apps, but the output summary at the end is explicit about infra:

"""
3 of these updates are provided through UA Infrastructure ESM.
"""

We should either count them separately, and issue separate sentences, or, as I would prefer, just remove "Infrastructure" from the sentence above. That avoids increasing the MOTD size by one line.

description: updated
description: updated
description: updated
Andreas Hasenack (ahasenack) wrote :

I uploaded the fixed package to the SRU queue.

An upload of update-notifier to trusty-proposed has been rejected from the upload queue for the following reason: "trusty is closed for SRUs unless absolutely required; should go to the ESM archive instead".

Andreas Hasenack (ahasenack) wrote :

This is available in trusty esm now:

  Version table:
     0.154.1ubuntu9 0
        500 https://esm.ubuntu.com/ubuntu/ trusty-infra-security/main amd64 Packages

Closing bug.

Changed in update-notifier (Ubuntu Trusty):
status: In Progress → Fix Released
Łukasz Zemczak (sil2100) wrote :

Already asked on IRC, but re-asking here for documentation purposes: how would it be tested for all the non-trusty series? Since the test case mentions using a trusty lxd to perform the tests. Is it possible to perform the same testing steps on xenial, bionic or focal (since we don't have ESM for those)? If not, what would be the acceptance criteria for this to be verified on those series?

Chad Smith (chad.smith) wrote :

Thanks again Łukasz,
I've updated the test script on this bug. We have a ppa:ua-client/proposed which does support trusty, xenial, bionic and focal which will allow us to exercise this changeset.

description: updated

Hello Chad, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in update-notifier (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed verification-needed-xenial
Chris Halse Rogers (raof) wrote :

Hello Chad, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in update-notifier (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed-bionic
Changed in update-notifier (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed-focal
Chris Halse Rogers (raof) wrote :

Hello Chad, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted update-notifier (3.192.1.8) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

update-notifier/3.192.1.8 (armhf, ppc64el, s390x, i386, amd64, arm64)
update-manager/1:18.04.11.13 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

All autopkgtests for the newly accepted update-notifier (3.168.11) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

update-notifier/3.168.11 (armhf, i386, arm64, amd64, s390x, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.
This report contains Public Security information  Edit
Everyone can see this security related information.

Other bug subscribers