Please backport fix for & in attributes

Bug #1780442 reported by Mario Limonciello on 2018-07-06
98
This bug affects 17 people
Affects Status Importance Assigned to Milestone
Fwupd
Fix Released
Unknown
appstream-glib (Ubuntu)
Undecided
Unassigned
Xenial
Medium
Robert Ancell
fwupd (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Unassigned

Bug Description

[Impact]

There are instances of fwupd being unable to run updates on certain devices on Ubuntu 16.04. due to a "&" in metadata.

[Test Case]

 * Try to perform an update on a 8bitdo affected device.

[Regression Potential]

 * Regressions would occur in metadata processing where the fwupd daemon wouldn't be able to process it.

[Other Info]

This was discussed here:
https://github.com/hughsie/fwupd/issues/565#issuecomment-402534337

This has been fixed in appstream-glib to prevent & in the metadata. This fix is already in 18.04 and just needs to be backported to 16.04.
https://github.com/hughsie/appstream-glib/commit/6048520484101df5d33f3c852c10640e630d20cf

Changed in appstream-glib (Ubuntu):
status: New → Fix Released
description: updated
Changed in fwupd:
status: Unknown → New
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in appstream-glib (Ubuntu Xenial):
status: New → Confirmed
Mario Limonciello (superm1) wrote :

Another upstream report of this was here:
https://github.com/hughsie/fwupd/issues/590

François Marier (fmarier) wrote :

From https://bugs.launchpad.net/ubuntu/+source/appstream-glib/+bug/1780520/comments/2:

"I have uploaded a fixed package to my PPA: https://launchpad.net/~fmarier/+archive/ubuntu/ppa?field.series_filter=xenial

Here's how I fixed the problem:

1. upgrade the appstream-glib package to the one in my PPA
2. delete /var/cache/app-info/xmls/fwupd.xml
3. run `fwupdmgr refresh` as root to re-download the XML file"

This is the patch I added to the package: https://bugs.launchpad.net/ubuntu/+source/appstream-glib/+bug/1780520/+attachment/5161135/+files/06_ampersand_in_attribute_values.patch

Is the step to delete old metadata actually required? That will complicate
the SRU if so..

On Wed, Jul 11, 2018, 13:25 François Marier <email address hidden> wrote:

> >From https://bugs.launchpad.net/ubuntu/+source/appstream-
> glib/+bug/1780520/comments/2:
>
> "I have uploaded a fixed package to my PPA:
>
> https://launchpad.net/~fmarier/+archive/ubuntu/ppa?field.series_filter=xenial
>
> Here's how I fixed the problem:
>
> 1. upgrade the appstream-glib package to the one in my PPA
> 2. delete /var/cache/app-info/xmls/fwupd.xml
> 3. run `fwupdmgr refresh` as root to re-download the XML file"
>
> This is the patch I added to the package:
> https://bugs.launchpad.net/ubuntu/+source/appstream-
>
> glib/+bug/1780520/+attachment/5161135/+files/06_ampersand_in_attribute_values.patch
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1780442
>
> Title:
> Please backport fix for & in attributes
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fwupd/+bug/1780442/+subscriptions
>

François Marier (fmarier) wrote :

I was still getting the error messages after upgrading the package because the cached file still had the unescaped ampersand in it.

It seems like `fwupdmgr refresh` doesn't re-generate the file if it hasn't changed on the server.

gumbeto (gumbeto) wrote :

It appears --force does not do it after all, I had to remove the file.

Changed in appstream-glib (Ubuntu Xenial):
assignee: nobody → Robert Ancell (robert-ancell)
importance: Undecided → Medium
status: Confirmed → Fix Committed
Robert Ancell (robert-ancell) wrote :

Uploaded fix in appstream-glib 0.5.13-1ubuntu6

Kamil (kamil-j-nizinski) wrote :

When will this fix be released?

Robert Ancell (robert-ancell) wrote :

It needs to go through the SRU process (https://wiki.ubuntu.com/StableReleaseUpdates) which can take a few weeks.

Alejandro Guerrero (alegu) wrote :

I want to make sure it's clear this is a high-impact regression. Right now I'm unable to use Software at all because of this, Software is empty and says "no software data found". Software is useless. Manually editing fwpd.xml or removing it and refreshing appstream restores functionality. For me personally this ain't such a big deal as I know how to workaround it, but normal users will be clueless and should not be expected having to wait weeks for restored functionality because of something as trivial as a unescaped ampersand.

Alejandro Guerrero (alegu) wrote :

Sorry it should say "fwupd.xml". Weird that you can't edit recent comments at Launchpad, maybe Ubuntu should consider switching to GitLab ;)

Hello Mario, or anyone else affected,

Accepted appstream-glib into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/appstream-glib/0.5.13-1ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed verification-needed-xenial
Robie Basak (racb) wrote :

Accepted since the fix will help anyone new impacted by this bug. But what happened to the upgrade path as mentioned in comments 4 and 5? Should the cache be deleted in postinst if upgrading up to a fixed version?

Please add this to Regression Potential and test for this during SRU verification.

gumbeto (gumbeto) wrote :

Hello,

The package name here is libappstream-glib8 (no package found by the name appstream-glib). I installed the new version:

$ apt-cache policy libappstream-glib8 | grep -i installed
  Installed: 0.5.13-1ubuntu6

... and I confirm it fixes the bug here, *under the following circumstances*:

1. I did reboot just after installing the new version. I am not sure whether this step is necessary, but I wanted to force a reload of the lib in memory, as it was in use by a couple of processes:

$ dpkg -L libappstream-glib8 | grep .so
/usr/lib/x86_64-linux-gnu/libappstream-glib.so.8.0.6
/usr/lib/x86_64-linux-gnu/libappstream-glib.so.8
$ lsof /usr/lib/x86_64-linux-gnu/libappstream-glib.so*
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
compiz 2096 **** mem REG 8,17 279256 1837580 /usr/lib/x86_64-linux-gnu/libappstream-glib.so.8.0.6
gnome-sof 2249 **** mem REG 8,17 279256 1837580 /usr/lib/x86_64-linux-gnu/libappstream-glib.so.8.0.6

2. It did not work until I removed the offending cache file first. Running `sudo fwupdmgr refresh --force` and `sudo appstreamcli refresh-cache --force --verbose` wouldn't update the file otherwise.

gumbeto (gumbeto) wrote :

(I am not adding the verification-done tag as updating the package without removing the cache was not enough to fix the issue)

Robert Ancell (robert-ancell) wrote :

gumbeto - I'd recommend opening a new bug about the issue for clients with the bad cache. This fix is still useful as it will stop new users hitting the problem. I've asked on https://github.com/hughsie/fwupd/issues/565 if upstream has a solution for that issue.

If you open a new bug please mark this one as verified.

gumbeto (gumbeto) wrote :

Hello, I agree this is very helpful already, but it seems it is still not a full fix for this issue yet. There is no other reason for removing and refreshing the cache than this bug, so I think opening a new one would only confuse things.

Mario Limonciello (superm1) wrote :

Per discussion upstream there is a commit needed for fwupd as well to make it not abort on the bad data. So the appstream-glib tasks should get a verification-done tag and flow through, but we also need to SRU fwupd with this commit: https://github.com/hughsie/fwupd/commit/fe1c4de5a4b178cae3e4e9325ea84ff85c4b1be3

Changed in fwupd (Ubuntu):
status: New → Fix Released
Mario Limonciello (superm1) wrote :

@Robert
I've uploaded a fwupd package with the designated above fix.

@gumbeto
Watch this bug for when an archive admin releases the fix and you can try it out by putting in some corrupted data back into the cache again (you can manually modify it to "corrupt" it).

gumbeto (gumbeto) wrote :

Sure, will try to do that tomorrow.

Timo Aaltonen (tjaalton) wrote :

Hello Mario, or anyone else affected,

Accepted fwupd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fwupd/0.8.3-0ubuntu4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in fwupd (Ubuntu Xenial):
status: New → Fix Committed
gumbeto (gumbeto) on 2018-08-17
tags: added: verification-done-xenial
removed: verification-needed-xenial
gumbeto (gumbeto) wrote :

@Mario et all, as a simple user I am not very familiar with bug tracking procedures for Ubuntu, but I understood the verification tag to be associated with the bug, not a package. It did not make much sense to me to mark the bug verified until there was a full fix. I suppose you can't be sure something will be part of a final fix until the whole is verified (e.g. a newer version of the same pkg might be needed).

Anyway, I just tested `fwupdmgr refresh` with both new packages and an old cache file with the unescaped ampersand. It worked perfectly here, so updated tag to verification-done-xenial

Łukasz Zemczak (sil2100) wrote :

Thanks gumbeto for the verification! For the future: it is also important for the verification to include package version numbers of what -proposed packages have been tested. Otherwise yes, waiting with performing the validation until all required packages are in -proposed is a good choice.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package appstream-glib - 0.5.13-1ubuntu6

---------------
appstream-glib (0.5.13-1ubuntu6) xenial; urgency=medium

  * debian/patches/0001-Never-include-in-attribute-values.patch:
    - Handle '&' in attribute values (LP: #1780442)

 -- Robert Ancell <email address hidden> Mon, 30 Jul 2018 11:43:02 +1200

Changed in appstream-glib (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for appstream-glib has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fwupd - 0.8.3-0ubuntu4

---------------
fwupd (0.8.3-0ubuntu4) xenial; urgency=medium

  * debian/patches 0001-Do-not-abort-startup-if-the-XML-metadata-file-is-inv.patch:
    - Fix a crash from bad metadata (LP: #1780442)

 -- Mario Limonciello <email address hidden> Thu, 16 Aug 2018 13:41:15 -0500

Changed in fwupd (Ubuntu Xenial):
status: Fix Committed → Fix Released
Changed in fwupd:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.