Showing esm update as installable when esm is disabled

Bug #1883315 reported by Andreas Hasenack
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
update-notifier (Ubuntu)
Fix Released
High
Chad Smith
Xenial
Fix Released
High
Lucas Albuquerque Medeiros de Moura
Bionic
Fix Released
High
Lucas Albuquerque Medeiros de Moura
Focal
Fix Released
High
Lucas Albuquerque Medeiros de Moura
Groovy
New
Undecided
Unassigned
Hirsute
Fix Released
High
Chad Smith
Impish
Fix Released
High
Chad Smith

Bug Description

[Impact]
when users are getting the message update-notifier message through apt-check they may find inconsistent behavior regarding ESM products. This is misleading since we will say to the users that they don't have ESM Infra, but they do have ESM infra packages that can be installed. This is poor marketing of our products

[Test case]

To reproduce the issue, you can:

1. Launch the following old version of a xenial container:
   lxc launch ubuntu:f4c4c60a6b752a381288ae72a1689a9da00f8e03b732c8d1b8a8fcd1a8890800 dev-x

2. Run apt update and install the updated version of update-notifier-common
3. Add the ubuntu-advantage-tools ppa:
   https://code.launchpad.net/~ua-client/+archive/ubuntu/daily
4. Install ubuntu-advantage-tools
5. Install the latest version of uaclient from the stable ppa:
   https://launchpad.net/~ua-client/+archive/ubuntu/stable/
6. Comment out all mentions of xenial-security in /etc/apt/source.list
7. Run apt update
8. Run /usr/lib/update-notifier/apt-check --human-readable
9. See a message like this:

UA Infra: Extended Security Maintenance (ESM) is not enabled.

256 packages can be updated.
5 of these updates are fixed through UA Infra: ESM.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

Enable UA Infra: ESM to receive 5 additional security updates.
See https://ubuntu.com/security/esm or run: sudo ua status

To verify that the error is fixed:

1.Perform all the stages above until step 8
2 Install the new update-notifier from this ppa:
  https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa
3. Run /usr/lib/update-notifier/apt-check --human-readable
4. See a message like this:

256 updates can be installed immediately.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

5. We are now only showing ESM infra specific message if the distro is ESM. To enforce that behavior, make the `is_esm_distro` function in `/usr/lub/update-notifier/apt-check` return True, then you will see this message:

UA Infra: Extended Security Maintenance (ESM) is not enabled.

256 updates can be installed immediately.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

5 additional security updates can be applied with UA Infra: ESM
Learn more about enabling UA Infra: ESM service at https://ubuntu.com/esm

That is now correct.

[Where problems could occur]

The changes in this package should only be seen when MOTD is getting a new message. If that script fails for some reason, it seems that MOTD will only not present the message, which is doesn't seem to be a system critical issue. Additionally, we would potentially have tracebacks in the update-notifier logs. Finally, if the logic is also incorrect, we would be displying incorrect ESM messages to the user. But since we are doing this now, as this bug shows, I don't think this is critical as well.

[Discussion]

With ESM Apps going to production soon, we have decided to update the messages delivered by update-notifier apt-check to address the package count of ESM Apps and the possibility of installing more upgrades if the user has ESM Apps disabled.

We are also updating other parts of the messaging as well. First, we only display ESM Infra status
on ESM distros. However, we will keep showing the ESM Infra package count on all of them.

For ESM Apps, we are only performing the alerts (For example, that you might have x packages updates if ESM Apps is installed) if the user is on a LTS distro.

Since we going to perform that change, we decided to also address this bit in the SRU, since it could harm the message we are delivering

[Original Report]
I came across a scenario where the output of `/usr/lib/update-notifier/apt-check --human-readable` is showing some (not all) esm updates as being installable when esm itself is disabled:

ubuntu@trusty-desktop:~$ sudo /usr/lib/update-notifier/apt-check --human-readable
UA Infrastructure Extended Security Maintenance (ESM) is not enabled.

456 updates can be installed immediately.
10 of these updates are provided through UA Infrastructure ESM.
378 of these updates are security updates.
To see these additional updates run: apt list --upgradable

Enable UA Infrastructure ESM to receive 127 additional security updates.
See https://ubuntu.com/advantage or run: sudo ua status

If you look carefully, you will see that it's contradicting itself by saying esm is enabled and disabled at the same time:
- 10 ESM updates can be installed immediately
- ESM is disabled, and if you enable ESM you will get 127 additional updates

I believe this comes from apt_check.py:253:

            # now check for security updates that are masked by a
            # canidate version from another repo (-proposed or -updates)
            for ver in pkg.version_list:
                if (inst_ver and apt_pkg.version_compare(ver.ver_str, inst_ver.ver_str) <= 0):
                    #print("skipping '%s' " % ver.VerStr)
                    continue
                if isESMUpgrade(ver):
                    esm_updates += 1
                if isSecurityUpgrade(ver):
                    security_updates += 1
                    break

I believe that is ignoring the fact that ESM is disabled. I added a pdb to check which package it was considering as an esm update, and the first response was dbus, which is in this peculiar state in the archive:
ubuntu@trusty-desktop:~$ apt-cache policy dbus
dbus:
  Installed: 1.6.18-0ubuntu4.3
  Candidate: 1.6.18-0ubuntu4.5
  Version table:
     1.6.18-0ubuntu4.5+esm1 0
       -32768 https://esm.ubuntu.com/ubuntu/ trusty-infra-security/main amd64 Packages
     1.6.18-0ubuntu4.5 0
        500 http://br.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
     1.6.18-0ubuntu4.4 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
 *** 1.6.18-0ubuntu4.3 0
        100 /var/lib/dpkg/status
     1.6.18-0ubuntu4 0
        500 http://br.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

Maybe we just need to guard that isESMUpgrade(ver) call with "if have_esm and isESMUpgrade(ver)"?

The other place in the code a bit up from the above which also increments esm_updates isn't run in this scenario, so the 10 packages must come from the check I highlighted above.

Other info:
update-notifier 0.154.1ubuntu8 from trusty-updates
ubuntu-advantage-tools 19.6~ubuntu14.04.4 from trusty-updates
ua is attached, but esm disabled:
ubuntu@trusty-desktop:~$ ua status
SERVICE ENTITLED STATUS DESCRIPTION
cc-eal yes n/a Common Criteria EAL2 Provisioning Packages
cis-audit no — Center for Internet Security Audit Tools
esm-infra yes disabled UA Infra: Extended Security Maintenance
fips yes n/a NIST-certified FIPS modules
fips-updates yes n/a Uncertified security updates to FIPS modules
livepatch yes disabled Canonical Livepatch service

Enable services with: ua enable <service>

     Account: <email address hidden>
Subscription: <email address hidden>

Related branches

Bryce Harrington (bryce)
Changed in update-notifier (Ubuntu Xenial):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :
Download full text (3.6 KiB)

[Impact]
when users are getting the message update-notifier message through apt-check they may find inconsistent behavior regarding ESM products. This is misleading since we will say to the users that they don't have ESM Infra, but they do have ESM infra packages that can be installed. This is poor marketing of our products

[Test case]

To reproduce the issue, you can:

1. Launch the following old version of a xenial container:
   lxc launch ubuntu:f4c4c60a6b752a381288ae72a1689a9da00f8e03b732c8d1b8a8fcd1a8890800 dev-x

2. Run apt update and install the updated version of update-notifier-common
3. Add the ubuntu-advantage-tools ppa:
   https://code.launchpad.net/~ua-client/+archive/ubuntu/daily
4. Install ubuntu-advantage-tools
5. Install the latest version of uaclient from the stable ppa:
   https://launchpad.net/~ua-client/+archive/ubuntu/stable/
6. Comment out all mentions of xenial-security in /etc/apt/source.list
7. Run apt update
8. Run /usr/lib/update-notifier/apt-check --human-readable
9. See a message like this:

UA Infra: Extended Security Maintenance (ESM) is not enabled.

256 packages can be updated.
5 of these updates are fixed through UA Infra: ESM.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

Enable UA Infra: ESM to receive 5 additional security updates.
See https://ubuntu.com/security/esm or run: sudo ua status

To verify that the error is fixed:

1.Perform all the stages above until step 8
2 Install the new update-notifier from this ppa:
  https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa
3. Run /usr/lib/update-notifier/apt-check --human-readable
4. See a message like this:

256 updates can be installed immediately.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

5. We are now only showing ESM infra specific message if the distro is ESM. To enforce that behavior, make the `is_esm_distro` function in `/usr/lub/update-notifier/apt-check` return True, then you will see this message:

UA Infra: Extended Security Maintenance (ESM) is not enabled.

256 updates can be installed immediately.
5 of these updates are security updates.
To see these additional updates run: apt list --upgradable

5 additional security updates can be applied with UA Infra: ESM
Learn more about enabling UA Infra: ESM service at https://ubuntu.com/esm

That is now correct.

[Where problems could occur]

The changes in this package should only be seen when MOTD is getting a new message. If that script fails for some reason, it seems that MOTD will only not present the message, which is doesn't seem to be a system critical issue. Additionally, we would potentially have tracebacks in the update-notifier logs. Finally, if the logic is also incorrect, we would be displying incorrect ESM messages to the user. But since we are doing this now, as this bug shows, I don't think this is critical as well.

[Discussion]

With ESM Apps going to production soon, we have decided to update the messages delivered by update-notifier apt-check to address the package count of ESM Apps and the possibility of installing more upgrades if the user has...

Read more...

Changed in update-notifier (Ubuntu Xenial):
assignee: nobody → Lucas Albuquerque Medeiros de Moura (lamoura)
Bryce Harrington (bryce)
Changed in update-notifier (Ubuntu Xenial):
status: Triaged → In Progress
Bryce Harrington (bryce)
Changed in update-notifier (Ubuntu Bionic):
assignee: nobody → Lucas Albuquerque Medeiros de Moura (lamoura)
Changed in update-notifier (Ubuntu Focal):
assignee: nobody → Lucas Albuquerque Medeiros de Moura (lamoura)
Changed in update-notifier (Ubuntu Hirsute):
assignee: nobody → Chad Smith (chad.smith)
Changed in update-notifier (Ubuntu Impish):
assignee: nobody → Chad Smith (chad.smith)
Bryce Harrington (bryce)
Changed in update-notifier (Ubuntu Bionic):
status: New → In Progress
Changed in update-notifier (Ubuntu Focal):
status: New → In Progress
Changed in update-notifier (Ubuntu Hirsute):
status: New → In Progress
Changed in update-notifier (Ubuntu Impish):
status: New → In Progress
Changed in update-notifier (Ubuntu Bionic):
importance: Undecided → High
Changed in update-notifier (Ubuntu Focal):
importance: Undecided → High
Changed in update-notifier (Ubuntu Hirsute):
importance: Undecided → High
Changed in update-notifier (Ubuntu Impish):
importance: Undecided → High
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Andreas, or anyone else affected,

Accepted update-notifier into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.40.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in update-notifier (Ubuntu Hirsute):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-hirsute
Changed in update-notifier (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Andreas, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in update-notifier (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Andreas, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Andreas, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in update-notifier (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (update-notifier/3.192.30.7)

All autopkgtests for the newly accepted update-notifier (3.192.30.7) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10.6 (armhf, ppc64el, amd64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

By reusing the same script in this launchpad bug:
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1924766

I can confirm that the xenial, bionic, focal and hirsute proposed packages are working as expected

tags: added: verification-done verification-done-bionic verification-done-focal verification-done-hirsute verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-hirsute verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.192.41

---------------
update-notifier (3.192.41) impish; urgency=medium

  [ Lucas Moura ]

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Chad Smith <email address hidden> Thu, 22 Apr 2021 17:47:19 -0600

Changed in update-notifier (Ubuntu Impish):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.192.40.1

---------------
update-notifier (3.192.40.1) hirsute; urgency=medium

  [ Lucas Moura ]
  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Chad Smith <email address hidden> Thu, 22 Apr 2021 17:47:19 -0600

Changed in update-notifier (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.192.30.7

---------------
update-notifier (3.192.30.7) focal; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * data/backend_helper.py:
    - fix pyflakes test
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Thu, 22 Apr 2021 18:56:22 -0300

Changed in update-notifier (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.192.1.10

---------------
update-notifier (3.192.1.10) bionic; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Thu, 22 Apr 2021 18:39:19 -0300

Changed in update-notifier (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package update-notifier - 3.168.14

---------------
update-notifier (3.168.14) xenial; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction betweem standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Tue, 20 Apr 2021 10:20:21 -0300

Changed in update-notifier (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers