/usr/lib/update-notifier/package-data-downloader:KeyError:/usr/lib/update-notifier/package-data-downloader@351:process_download_requests:trigger_update_notifier:substitute:convert

[errors.ubuntu.com bug bridge]

[Original Description]

The Ubuntu Error Tracker has been receiving reports about a problem regarding update-notifier. This problem was most recently seen with package version 3.192.60, the problem page at https://errors.ubuntu.com/problem/09c9352b62a3d0ae149fb2c93dbd80a2a84d4053 contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker and are a software developer, you can request it at http://forms.canonical.com/reports/

[Impact]

If there is an error on the package-data-downloader script and the user is using an affected locale, this traceback will appear and prevent the user from seeing the real issue that is affecting the script.

[Test Case]
You can replicate this issue by running the following script:

---------------------------------
#!/bin/bash
set -e

series=$1
name=$series-dev

cleanup () {
  lxc delete $name --force
}

on_err () {
  echo -e "Test Failed"
  cleanup
  exit 1
}

trap on_err ERR

lxc launch ubuntu-daily:$series $name
sleep 10

echo "#########################################"
echo "Installing latest version of update-notifier"
lxc exec $name -- apt-get update > /dev/null
lxc exec $name -- sudo apt-get install update-notifier-common -y > /dev/null
lxc exec $name -- apt-cache policy update-notifier-common
echo -e "#########################################\n"

echo "#########################################"
echo "Change locales to welsh"
lxc exec $name -- sh -c "DEBIAN_FRONTEND=noninteractive sudo apt-get install language-pack-cy -y > /dev/null"
lxc exec $name -- sudo update-locale LANG=cy_GB.UTF-8
lxc exec $name -- sudo reboot
sleep 5
lxc config set $name environment.LC_ALL=cy_GB.UTF-8
echo -e "#########################################\n"

echo "#########################################"
echo "Installing fonts package (verify stacktrace)"
lxc exec $name -- apt-get update > /dev/null
lxc exec $name -- apt-get -y --download-only install ttf-mscorefonts-installer
lxc config device add $name eth0 none
pkg_name=$(lxc exec $name -- ls /var/cache/apt/archives/ | grep ttf)
lxc exec $name -- sh -c "cd /var/cache/apt/archives; echo ttf-mscorefonts-installer msttcorefonts/accepted-mscorefonts-eula select true | sudo debconf-set-selections"
lxc exec $name -- sh -c "dpkg -i /var/cache/apt/archives/$pkg_name || true"
lxc exec $name -- apt purge ttf-mscorefonts-installer -y > /dev/null
echo -e "#########################################\n"

lxc config device remove $name eth0
lxc exec $name -- sudo reboot
sleep 5

echo "#########################################"
echo "Update update-notifier package to proposed version"
lxc exec $name -- sh -c "echo \"deb http://archive.ubuntu.com/ubuntu $series-proposed main\" | tee /etc/apt/sources.list.d/proposed.list"
lxc exec $name -- sudo apt-get update > /dev/null
lxc exec $name -- sudo apt-get install update-notifier-common -y > /dev/null
lxc exec $name -- apt-cache policy update-notifier-common
echo -e "#########################################\n"

echo "#########################################"
echo "Installing fonts package (verify no error)"
lxc config device add $name eth0 none
lxc exec $name -- sh -c "cd /var/cache/apt/archives; echo ttf-mscorefonts-installer msttcorefonts/accepted-mscorefonts-eula select true | sudo debconf-set-selections"
lxc exec $name -- sh -c "dpkg -i /var/cache/apt/archives/$pkg_name || true"
echo -e "#########################################\n"

echo "#########################################"
echo "Verify that file created by package-data-downloader has the \$package variable replaced"
if lxc exec $name -- grep -Fq "\$package" /var/lib/update-notifier/user.d/data-downloads-failed; then
    echo "FAILURE: \$package variable found"
else
    echo "SUCCESS: No \$package variable found"
fi

if lxc exec $name -- grep -Fq "ttf-mscorefonts-installer" /var/lib/update-notifier/user.d/data-downloads-failed; then
    echo "SUCCESS: ttf-mscorefonts-installer appears in the file"
else
    echo "FAILURE: ttf-mscorefonts-installer does not appear in the file"
fi

echo -e "#########################################\n"
cleanup
--------------------------------------------

Once the new package is in proposed, we can re-use this script to verify that the traceback is no longer appearing.

[Discussion]

This error is being caused by an incorrectly translated variable in a template file. This template file is being used by the package-data-downloader script, which is delivered by the update-notifier-common
package. If the user is using one of the affected locales (i.e. cy_GB.UTF-8) and the user installs a package that triggers it (i.e. ttf-mscorefonts-installer) and the script fail somehow, the user will be presented with this stacktrace error.

The fix for this is to simply not translate the variable name in the template file.

[Changelog]

  * po/*.po: do not translate template variable needed for
    package-data-downloader script

[Andreas Hasenack (ahasenack)]

The traceback seems to suggests that a string that shouldn't have been translated, was:

Traceback (most recent call last):
  File "/usr/lib/update-notifier/package-data-downloader", line 351, in <module>
    process_download_requests()
  File "/usr/lib/update-notifier/package-data-downloader", line 345, in process_download_requests
    trigger_update_notifier(our_failures)
  File "/usr/lib/update-notifier/package-data-downloader", line 156, in trigger_update_notifier
    string.Template(input).substitute(
  File "/usr/lib/python3.10/string.py", line 121, in substitute
    return self.pattern.sub(convert, self.template)
  File "/usr/lib/python3.10/string.py", line 114, in convert
    return str(mapping[named])
KeyError: 'pecynnau'

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas Hasenack (ahasenack)]

Hello errors.ubuntu.com, or anyone else affected,

Accepted update-notifier into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.59.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello errors.ubuntu.com, or anyone else affected,

Accepted update-notifier into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.54.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello errors.ubuntu.com, or anyone else affected,

Accepted update-notifier into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.30.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.30.16) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

update-notifier/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted update-notifier (3.192.59.2) for kinetic have finished running.
The following regressions have been reported in tests triggered by the package:

update-notifier/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/kinetic/update_excuses.html#update-notifier

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Andreas Hasenack (ahasenack)]

Hello errors.ubuntu.com, or anyone else affected,

Accepted update-notifier into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.192.1.18 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello errors.ubuntu.com, or anyone else affected,

Accepted update-notifier into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-notifier/3.168.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Lucas Albuquerque Medeiros de Moura (lamoura)]

I have tested this fix with the script mentioned in the test section and everything is worked as expected

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.62

---------------
update-notifier (3.192.62) lunar; urgency=medium

  [ Lucas Moura ]
  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

 -- Sebastien Bacher <email address hidden> Tue, 24 Jan 2023 21:10:45 +0100

[Andreas Hasenack (ahasenack)]

I verified the attached logs and am satisfied that they show the executed planned test case, and that the results are correct.

The package built correctly in all architectures and Ubuntu releases it was meant for.

There are no DEP8 regressions.

There is no SRU freeze ongoing at the moment.

There is a halted phasing on the previous update, which this upload here is fixing.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.59.2

---------------
update-notifier (3.192.59.2) kinetic; urgency=medium

  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 16:11:49 -0300

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for update-notifier has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.54.5

---------------
update-notifier (3.192.54.5) jammy; urgency=medium

  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

update-notifier (3.192.54.4) jammy; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 15:50:39 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.30.16

---------------
update-notifier (3.192.30.16) focal; urgency=medium

  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

update-notifier (3.192.30.15) focal; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 16:17:53 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.192.1.18

---------------
update-notifier (3.192.1.18) bionic; urgency=medium

  * po/*.po: do no translate template variable needed for
    package-data-downloader script (LP: #2003543)
  * Bring generated autoconf files from the archive package (LP: #1999897)

update-notifier (3.192.1.16) bionic; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

 -- Lucas Moura <email address hidden> Fri, 20 Jan 2023 16:25:17 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package update-notifier - 3.168.19

---------------
update-notifier (3.168.19) xenial; urgency=medium

  [ Renan Rodrigo Barbosa]
  * Rely on the Pro Client apt esm cache to check for esm updates
    (LP: #2002168)

  [ Lucas Moura ]
  * po/*.po: do not translate template variable needed for
    package-data-downloader script (LP: #2003543)

 -- Christian Ehrhardt <email address hidden> Thu, 19 Jan 2023 15:21:16 +0100