[Impact]
* The recent update of the Ubuntu asgen instance yielded it to generate modern metadata. While we could tell asgen to generate old-style metadata, fixing appstream is the better idea, since we already applied similar patches to appstream-glib (and having this package hold back updates is a bad idea). Currently, the unknown metadata is considered to be broken by appstream and it displays a warning about that issue during an `apt update`
* The attached patch includes a second change to make AppStream more robust against broken YAML data, ensuring LP #1579712 can not be triggered on purpose or due to broken disks creating damaged YAML structure.
[Test Case 1]
* apt update
* Check for errors about metadata being ignored
[Test Case 2]
* appstreamcli refresh --force
* Check for errors about metadata being ignored
* Check if we ignored data that should be present
(search for it via `appstreamcli search <term>`)
* Run `appstreamcli status` and see if we still have information on the same amount of components as prior to the update.
[Regression Potential]
* Medium; The works case would be KDE Discover and GNOME Software loosing information on available software, but this risk is very low and the patch to fix this bug is very simple and has almost no regression potential. The YAML parser fix patch is a bit more complex, but should have no sideeffects.
[Original report]
Periodically I get emails from a machine with the subject:
Cron <root@Jenkins> if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi
And a body of:
AppStream cache update completed, but some metadata was ignored due to errors.
I have a machine monitored by Munin. One of the plugins used on that machine is /etc/munin/plugins/apt which periodically counts the number of packets that can be upgraded. This is achieved by a crontab entry installed from the package which looks like this:
#
# cron-jobs for munin-node
#
MAILTO=root
# If the APT plugin is enabled, update packages databases approx. once
# an hour (12 invokations an hour, 1 in 12 chance that the update will
# happen), but ensure that there will never be more than two hour (7200
# seconds) interval between updates..
*/5 * * * * root if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi
For many months everything has been running smoothly. On the 23rd November 2016 I started getting the "AppStream cache update completed, but some metadata was ignored due to errors." messages at a rate of about ten a day.
I did a quick Google search which turned up a handful of bugs from earlier this year, such as
Bug1575248. That is marked as Fixed Released though, so I am raising this new bug.
If there are any further diagnostic details I can provide, please let me know.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: appstream 0.9.4-1ubuntu1
ProcVersionSignature: Ubuntu 4.4.0-47.68-generic 4.4.24
Uname: Linux 4.4.0-47-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.1
Architecture: amd64
Date: Thu Nov 24 09:57:18 2016
InstallationDate: Installed on 2013-05-08 (1295 days ago)
InstallationMedia: Ubuntu 13.10 "Saucy Salamander" - Alpha amd64 (20130508)
JournalErrors:
-- Logs begin at Wed 2016-11-23 19:25:01 GMT, end at Thu 2016-11-24 09:57:24 GMT. --
Nov 24 02:22:17 hostname systemd-tmpfiles[27330]: [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
ProcEnviron:
TERM=xterm-256color
SHELL=/bin/bash
PATH=(custom, no user)
LANG=en_GB.UTF-8
LANGUAGE=en_GB:en
SourcePackage: appstream
UpgradeStatus: Upgraded to xenial on 2016-08-24 (91 days ago)
Status changed to 'Confirmed' because the bug affects multiple users.