Empty MOTD on up-to-date systems (includes fix)

[Aaron Thompson]

[Impact]
On a system without updates to be installed, not showing any message in motd feels that something went wrong in the backend. Also, currently we are printing unwanted white spaces in that scenario, which is not a good user experience here.

[Test case]

To reproduce the issue, you can:

1. Launch a xenial container
2. Remove ubuntu-advantage-tools from the system
   This package will make update-notifier show more messages which are not needed in that test
3. Run an apt update
4. Install update-notifier
5. Run apt upgrade
6. Run /usr/lib/update-notifier/apt_check.py --human-readable
7. Verify we deliver an empty message

To verify that the error is fixed:

1. Run the past scenario until step 4
2. Install the new update-notifier from this ppa:
https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa/
3. Run `/usr/lib/update-notifier/apt-check --human-readable`
4. Verify that the command print the following message:

   0 updates can be applied immediately.

[Where problems could occur]

We are bringing back a behavior that was already present in the package, but removed in the last version of it. We will now be impacting users that may not be relying in that message anymore. But we believe this should not be a huge issue if that happens, since users were already relying on the old behavior.

[Discussion]
Currently, motd will not output a message if the system does not have any updates to apply. However, this is not ideal since this may cause the impression that there is something wrong with motd or the system. Also, the empty message we produce in that scenario is not completely empty, it does contain some white spaces on it, which is completely unnecessary.

Because of that, we are bringing back the behavior of printing the message:

0 updates can be applied immediately.

If no updates need to be applied in the system.

[Original description]

Starting with update-notifier-common 3.192.30.7 (focal), my up-to-date systems are showing this MOTD on login:

Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-72-generic x86_64)

user@host:~$

I've attached a patch that fixes this issue and adds more test cases.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "0001-apt_check.py-Fix-empty-output-in-the-no-ESM-up-to-da.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Lucas Albuquerque Medeiros de Moura (lamoura)]

Thank you for raising this. The change was made intentionally to reduce noise to users when there wasn't anything to act upon. We're always keeping an eye out for improving things and will keep this open as we discuss ways we can make things better.

[Daniel (g-ubuntuone-i)]

It looks weird without the update notification stuff and the large(ish) whitespace instead, can something be done about that? Alternatively, is it possible to make the suppression of 'no info' optional?

[Aaron Thompson (dev-aaront-org)]

Thanks for the response Lucas. I would prefer to see something to the effect that the system is up to date, because that at least tells me that update-notifier-common is there and working. But if the decision is to not write anything in that case, then I strongly agree with Daniel. The 3 blank lines are really noticeable and they make it look like something went wrong. It looks like something is supposed to be there but isn't. IMO there should be 1 or 0 blank lines. Anything more and it looks like a potential bug.

I would suggest that the tests from my patch (or similar) be added to the codebase, updated with the intended output of course. Such tests would make it clear that this change was intentional and not accidental.

[Daniel (g-ubuntuone-i)]

Seconding what Aaron said, it looks weird and makes it seem that something went wrong, causing me to wrack my brain and try to figure out if I broke something across multiple servers, lol.

[Andrew Moore (andrewmoore)]

I agree with Aaron and Daniel. I have a custom MOTD generation script that uses the output of updater-notifier `head -qn 2 /var/lib/update-notifier/updates-available`. Not having any text in that file if there are no updates available breaks the layout of the MOTD output.

Regardless of my custom setup, having the blank lines in the default MOTD generated by `/etc/motd.d/90-updates-available` does make me think something is broken. It's nice to have the validation of "0 packages can be updated", IMO.

[Tony Vroon (tonyvroon)]

There is still value in a "you are fully up to date" message, and I would expect such a behaviour change to be optional. This has not been announced and I find myself unable to opt out. Offering users more choice is great, but, I have no choices here. That doesn't sit right.

[Lucas Albuquerque Medeiros de Moura (lamoura)]

https://code.launchpad.net/~lamoura/update-notifier/+git/update-notifier/+merge/402163

[Lucas Albuquerque Medeiros de Moura (lamoura)]

SRU template verification

[Impact]
On a system without updates to be installed, not showing any message in motd feels that something went wrong in the backend. Also, currently we are printing unwanted white spaces in that scenario, which is not a good user experience here.

[Test case]

To reproduce the issue, you can:

1. Launch a xenial container
2. Remove ubuntu-advantage-tools from the system
   This package will make update-notifier show more messages which are not needed in that test
3. Run an apt update
4. Install update-notifier
5. Run apt upgrade
6. Run /usr/lib/update-notifier/apt_check.py --human-readable
7. Verify we deliver an empty message

To verify that the error is fixed:

1. Run the past scenario until step 4
2. Install the new update-notifier from this ppa:
https://launchpad.net/~lamoura/+archive/ubuntu/update-notifier-test-ppa/
3. Run `/usr/lib/update-notifier/apt-check --human-readable`
4. Verify that the command print the following message:

   0 updates can be applied immediately.

[Where problems could occur]

We are bringing back a behavior that was already present in the package, but removed in the last version of it. We will now be impacting users that may not be relying in that message anymore. But we believe this should not be a huge issue if that happens, since users were already relying on the old behavior.

[Discussion]
Currently, motd will not output a message if the system does not have any updates to apply. However, this is not ideal since this may cause the impression that there is something wrong with motd or the system. Also, the empty message we produce in that scenario is not completely empty, it does contain some white spaces on it, which is completely unnecessary.

Because of that, we are bringing back the behavior of printing the message:

0 updates can be applied immediately.

If no updates need to be applied in the system.

[Brian Murray (brian-murray)]

This fix is currently stuck in -proposed for impish but should be released shortly.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.44

---------------
update-notifier (3.192.44) impish; urgency=medium

  [ Lucas Moura ]
  * data/apt_check.py
    - Show esm-apps status header when running on LTS distro that
      has not yet entered Extended Security Maintenance

 -- Brian Murray <email address hidden> Thu, 13 May 2021 14:47:01 -0700

[Łukasz Zemczak (sil2100)]

Hello Aaron, or anyone else affected,

Accepted update-notifier into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.40.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Aaron, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Aaron, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Łukasz Zemczak (sil2100)]

Hello Aaron, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.15 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.40.3) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:21.04.10 (amd64, s390x, armhf, arm64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.30.8) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:20.04.10.7 (s390x, amd64, ppc64el, armhf, arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.1.11) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

update-manager/1:18.04.11.13 (i386, s390x, arm64, armhf, ppc64el, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Lucas Albuquerque Medeiros de Moura (lamoura)]

I can confirm that the xenial, bionic, focal and hirsute packages are working as expected.

To perform the verification I have used the following script:

--------------------------------------
#!/bin/sh
set -x

series=xenial
name=$series-dev

function generate_motd_message() {
    message=$1
    echo "-----------------------------------------------------------"
    echo $message
    lxc exec $name -- /usr/lib/update-notifier/update-motd-updates-available --force
    lxc exec $name -- update-motd
    echo "-----------------------------------------------------------"
}

function turn_distro_into_esm_mode() {
    # guarantee that xenial distro is on ESM mode
    lxc exec $name -- sed -i 's/is_esm_supported and is_not_currently_supported/True #comment/' /usr/lib/update-notifier/apt-check
}

function turn_distro_into_non_esm_mode() {
    # guarantee that xenial distro is on ESM mode
    lxc exec $name -- sed -i 's/True #comment/False/' /usr/lib/update-notifier/apt-check
}

function setup_update_notifier() {
    lxc exec $name -- sh -c "cat <<EOF >/etc/apt/sources.list.d/ubuntu-$series-proposed.list
    deb http://archive.ubuntu.com/ubuntu/ $series-proposed restricted main multiverse universe"
    lxc exec $name -- apt update
    lxc exec $name -- sh -c "apt install update-notifier -yq > /dev/null"
}

function install_ua_from_branch() {
    lxc exec $name --cwd /var/tmp/uac -- apt-get update
    lxc exec $name --cwd /var/tmp/uac -- apt-get install make
    lxc exec $name -- git clone https://github.com/canonical/ubuntu-advantage-client.git /var/tmp/uac
    lxc exec $name --cwd /var/tmp/uac -- sh -c "make deps > /dev/null"
    lxc exec $name --cwd /var/tmp/uac -- sh -c "DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -us -uc > /dev/null"
    lxc exec $name -- apt-get remove ubuntu-advantage-tools --assume-yes
    lxc exec $name --cwd /var/tmp/uac -- dpkg -i /var/tmp/ubuntu-advantage-tools_27.0_amd64.deb
    lxc exec $name -- ua version
}

function install_ua() {
    lxc exec $name -- add-apt-repository ppa:ua-client/daily -y
    lxc exec $name -- sudo apt-get update
    lxc exec $name -- sudo apt-get install ubuntu-advantage-tools -y
    lxc exec $name -- ua version
    lxc exec $name -- sudo apt-get update
}

function ua_disable_esm_apps() {
    lxc exec $name -- sudo ua disable esm-apps
}

function install_all_upgrades() {
    lxc exec $name -- sudo sh -c "apt update > /dev/null"
    lxc exec $name -- sudo sh -c "apt upgrade -y > /dev/null"
}

function ua_attach() {
    lxc exec $name -- sed -i 's/contracts.can/contracts.staging.can/' /etc/ubuntu-advantage/uaclient.conf
    lxc exec $name -- ua attach $UACLIENT_BEHAVE_CONTRACT_TOKEN_STAGING
}

function ua_detach() {
    lxc exec $name -- ua detach --assume-yes
}

function generate_ua_motd_messages() {
    lxc exec $name -- python3 /usr/lib/ubuntu-advantage/ua_update_messaging.py
}

function install_update_motd() {
    lxc exec $name -- apt install update-motd -yq
}

function turn_esm_apps_into_non_beta() {
    lxc exec $name -- sh -c "echo 'features:\n allow_beta: true' >> /etc/ubuntu-advantage/uaclient.conf"
}

function update_contract_effectiveto() {
    operation=$1
    num_days=$2
    replace_...

[Lucas Albuquerque Medeiros de Moura (lamoura)]

Also, regarding the autopkgtest failures we are seeing, they seem related to this update-manager bug:
https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/1929865

[Łukasz Zemczak (sil2100)]

Hey Lucas! What package versions did you use? Were those from the ubuntu archives? Since I see the instructions still mention using the PPA while for testing we need to use the archive packages.

[Lucas Albuquerque Medeiros de Moura (lamoura)]

Hi Lukasz, those instructions were created before the package was uploaded into ṕroposed. In the script I have used to test this bug, I have used the proposed package.

But if needed, I can update the instructions to use the proposed packages instead

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.40.3

---------------
update-notifier (3.192.40.3) hirsute; urgency=medium

  * debian/control: add lsb-release to the update-notifier-common depends as
    it is used by apt-cdrom-check. (LP: #1927996)
  * data/apt_check.py
    - Only show esm-apps alerts on lts and non-esm distros (LP: #1926990)
    - Update esm-infra alert for distros on ESM mode
    - Show message with number of upgradable packages even if that
      number is zero (LP: #1926819)
    - Get distro name and version directly from /etc/os-release
    - Fallback to lsb_release command if there are any problems using
      /etc/os-release as source of information
    - Show esm-apps status header when running on LTS distro that
      has not yet entered Extended Security Maintenance

 -- Lucas Moura <email address hidden> Fri, 14 May 2021 16:41:27 -0300

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.30.8

---------------
update-notifier (3.192.30.8) focal; urgency=medium

  * debian/control: add lsb-release to the update-notifier-common depends as
    it is used by apt-cdrom-check. (LP: #1927996)
  * data/apt_check.py
    - Only show esm-apps alerts on lts and non-esm distros (LP: #1926990)
    - Update esm-infra alert for distros on ESM mode
    - Show message with number of upgradable packages even if that
      number is zero (LP: #1926819)
    - Get distro name and version directly from /etc/os-release
    - Fallback to lsb_release command if there are any problems using
      /etc/os-release as source of information
    - Show esm-apps status header when running on LTS distro that
      has not yet entered Extended Security Maintenance

 -- Lucas Moura <email address hidden> Fri, 14 May 2021 16:02:18 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.1.11

---------------
update-notifier (3.192.1.11) bionic; urgency=medium

  * debian/control: add lsb-release to the update-notifier-common depends as
    it is used by apt-cdrom-check. (LP: #1927996)
  * data/apt_check.py
    - Only show esm-apps alerts on lts and non-esm distros (LP: #1926990)
    - Update esm-infra alert for distros on ESM mode
    - Show message with number of upgradable packages even if that
      number is zero (LP: #1926819)
    - Get distro name and version directly from /etc/os-release
    - Fallback to lsb_release command if there are any problems using
      /etc/os-release as source of information
    - Show esm-apps status header when running on LTS distro that
      has not yet entered Extended Security Maintenance

 -- Lucas Moura <email address hidden> Fri, 14 May 2021 15:43:17 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.168.15

---------------
update-notifier (3.168.15) xenial; urgency=medium

  * debian/control: add lsb-release to the update-notifier-common depends as
    it is used by apt-cdrom-check. (LP: #1927996)
  * data/apt_check.py
    - Only show esm-apps alerts on lts and non-esm distros (LP: #1926990)
    - Update esm-infra alert for distros on ESM mode
    - Show message with number of upgradable packages even if that
      number is zero (LP: #1926819)
    - Get distro name and version directly from /etc/os-release
    - Fallback to lsb_release command if there are any problems using
      /etc/os-release as source of information
    - Show esm-apps status header when running on LTS distro that
      has not yet entered Extended Security Maintenance

 -- Lucas Moura <email address hidden> Fri, 14 May 2021 15:00:04 -0300