standard security upgrade counts should not include ESM packages

[Lucas Albuquerque Medeiros de Moura]

[Impact]
When users are looking at MOTD messages, they might find the text confusing, since we don't explicitly say that the security updates count are taking into consideration both standard security pockets and ESM pockets.

[Test Case]
1. Launch the a xenial container
2. Add the ubuntu-advantage-tools ppa:
   https://code.launchpad.net/~ua-client/+archive/ubuntu/daily
3. Install ubuntu-advantage-tools
4. Attach to ua subscription
5. Comment out all mentions of xenial-security/xenial-updates in /etc/apt/source.list
6. Run apt update
7. Install libkrad0:
   apt install libkrad0=1.13.2+dfsg-5
8. Run /usr/lib/update-notifier/apt-check --human-readable
9. See a message like this:

UA Infra: Extended Security Maintenance (ESM) is not enabled. Install the latest version of uaclient from the stable ppa:
   https://launchpad.net/~ua-client/+archive/ubuntu/stable/d.

UA Infra: Extended Security Maintenance (ESM) is enabled.

3 packages can be updated.
1 of these updates is fixed through UA Infra: ESM.
1 of these updates is a security update.
To see these additional updates run: apt list --upgradable

To verify that the error is fixed:

1.Perform all the stages above until step 7
2. Bring back xenial-security on source.list (we need because of the python3-distro-info dependency of update-notifier-common)
3 Install the new update-notifier from this ppa:
  https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa
4. Remove xenial-security from source.list again
5. Run /usr/lib/update-notifier/apt-check --human-readable and see a message like this:

UA Infra: Extended Security Maintenance (ESM) is enabled.

4 updates can be installed immediately.
1 of these updates are UA Infra: ESM security updates.
To see these additional updates run: apt list --upgradable

That is now correct.

[Where problems could occur]

The changes in this package should only be seen when MOTD is getting a new message. If that script fails for some reason, it seems that MOTD will only not present the message, which is doesn't seem to be a system critical issue. Additionally, we would potentially have tracebacks in the update-notifier logs. Finally, if the logic is also incorrect, we would be displaying incorrect standard security messages to the user.

[Discussion]
Currently, we treat the upgrades coming from standard security pocket and ESM service with the same packaging count. This could be confusing, since we don't point that out in the current message that we have:

5 updates can be installed immediately.
5 of these updates are provide through UA Infrastructure ESM
5 of these updates are security updates

We believe this will be better if the message stated:
5 updates can be installed immediately.
5 of these updates are provide through UA Infrastructure ESM

And if we had a situation like that:

10 updates can be installed immediately.
5 of these updates are provide through UA Infrastructure ESM
8 of these updates are security updates

We would change it to:

10 updates can be installed immediately.
5 of these updates are provide through UA Infrastructure ESM
3 of these updates are standard security updates

[Brian Murray (brian-murray)]

Hello Lucas, or anyone else affected,

Accepted update-notifier into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.40.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Lucas, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Lucas, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello Lucas, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.14 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.30.7) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10.6 (armhf, ppc64el, amd64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Lucas Albuquerque Medeiros de Moura (lamoura)]

By reusing the same script in this launchpad bug:
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1924766

I can confirm that the xenial, bionic, focal and hirsute proposed packages are working as expected

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.41

---------------
update-notifier (3.192.41) impish; urgency=medium

  [ Lucas Moura ]

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Chad Smith <email address hidden> Thu, 22 Apr 2021 17:47:19 -0600

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.40.1

---------------
update-notifier (3.192.40.1) hirsute; urgency=medium

  [ Lucas Moura ]
  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Chad Smith <email address hidden> Thu, 22 Apr 2021 17:47:19 -0600

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.30.7

---------------
update-notifier (3.192.30.7) focal; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * data/backend_helper.py:
    - fix pyflakes test
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Thu, 22 Apr 2021 18:56:22 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.1.10

---------------
update-notifier (3.192.1.10) bionic; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction between standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Thu, 22 Apr 2021 18:39:19 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.168.14

---------------
update-notifier (3.168.14) xenial; urgency=medium

  * data/apt_check.py:
    - Add support to handle packages from ESM Apps in addition to ESM Infra
      and only display alerts if the distro is ESM. (LP: #1924766)
    - Do not display a count of ESM packages if the system does not have ESM
      enabled. (LP: #1883315)
    - Make distinction betweem standard security updates and ESM updates
      when performing package counts. (LP: #1926208)
    - use 'applied' instead of 'installed', redact 0 of these updates are
      security updates, and correct singular messages
  * debian/control: Add a dependency on python3-distro-info.

 -- Lucas Moura <email address hidden> Tue, 20 Apr 2021 10:20:21 -0300

[Brian Murray (brian-murray)]

Ubuntu 20.10 (Groovy Gorilla) has reached end of life, so this bug will not be fixed for that specific release.