Flatpak related refs are not installed when using GNOME Software on Ubuntu 18.04

Bug #1754864 reported by AsciiWolf on 2018-03-10
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
gnome-software (Ubuntu)
Low
Unassigned
Bionic
Low
Andrew Hayzen

Bug Description

When installing a Flatpak application that depends on some runtimes and they depend on some additional runtime extensions (related refs) using GNOME Software on Ubuntu 18.04, the runtime extensions are not installed. They are installed if you run "flatpak update" from Terminal after the installation. This issue should be already fixed in GNOME Software upstream so I believe that this is a downstream Ubuntu issue. Please, fix this.

Steps to Reproduce:
1. Use the latest, fully updated Ubuntu 18.04 image.
2. Install "flatpak" and "gnome-software-plugin-flatpak" packages.
3. Restart the system.
4. Download and add the Flathub repo file: https://dl.flathub.org/repo/flathub.flatpakrepo
5. Search and install the "GNOME Web" Flatpak package using GNOME Software.
6. Run the installed "Web" application.

Actual results:
GNOME Web is started with the default (Adwaita) theme (and without the correct locale if you use non-English system).

Expected results:
GNOME Web is started with the correct (Ubuntu) theme (and with the correct locale if you use non-English system).

Related branches

Jeremy Bicha (jbicha) wrote :

> This issue should be already fixed in GNOME Software upstream so I believe that this is a downstream Ubuntu issue.

Why do you think this?

AsciiWolf (asciiwolf) wrote :

Upstream issue (but I think this is an Ubuntu issue, not upstream one): https://gitlab.gnome.org/GNOME/gnome-software/issues/324

Jeremy Bicha (jbicha) wrote :

> It should be fixed in: https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/2

That particular commit is already in gnome-software 3.27.92.

Changed in gnome-software (Ubuntu):
importance: Undecided → Low
AsciiWolf (asciiwolf) wrote :

So, I have just tested this on the latest Fedora Rawhide and experienced the same issue as on Ubuntu 18.04 so it looks like that this is really an upstream issue.

Andrew Hayzen (ahayzen) on 2018-04-26
tags: added: flatpak-usability
AsciiWolf (asciiwolf) wrote :

https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/48

Here is a Merge Request that fixes the issue. It will hopefully be merged into the stable (gnome-3-28) branch in upcoming weeks.

AsciiWolf (asciiwolf) wrote :

The issue was fixed in the current master (gnome-3-30) branch. It still needs to be fixed in the older branch that Ubuntu 18.04 uses.

https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/58

Here is the actual Merge Request for the gnome-3-28 that is still waiting to be merged.

AsciiWolf (asciiwolf) wrote :

The issue is unfortunately still not fully fixed in master (see https://gitlab.gnome.org/GNOME/gnome-software/issues/235#note_280405) and is not fixed at all in the older branch (!58 was still not merged).

AsciiWolf (asciiwolf) on 2018-08-08
description: updated
Olivier Tilloy (osomon) wrote :

Curiously I cannot find "GNOME Web" when searching flathub from my cosmic VM. However I installed org.gnome.Eolie from GNOME Software and observed that it is started with Adwaita and an English locale, whereas the system theme is Ambiance and locale is French.
When installing from the CLI ("flatpak install flathub org.gnome.Eolie"), the theme is Ambiance as expected. The language remains English, maybe a separate issue?

Changed in gnome-software (Ubuntu):
status: New → Confirmed
Andrew Hayzen (ahayzen) wrote :

@osomon, hey o/ :-)

GNOME Web looks like it has been archived on Flathub https://github.com/flathub/org.gnome.Epiphany and removed from the store for now. I believe this was due to security issues, which will hopefully be solved with the new freedesktop runtime.

Eolie, looks like there are other people stating that the languages aren't working, so might just be that the translations aren't available for the app. (probably discuss with upstream)

Yes runtime extensions are not installed automatically with gnome software 3.28. Eg themes, extra drivers, codecs etc. This works with the CLI as it had extra logic to deal with automatically installing such extensions. Now with newer versions of libflatpak this logic is included as a "FlatpakTransaction API", GNOME Software 3.30 has a new flatpak plugin and will make use of this - resolving the issue for cosmic onwards.

However, there was also code submitted upstream which added this logic to the old GNOME Software flatpak plugin to install these extensions automatically - resolving the issue for 3.28 users. If this is approved and lands into a 3.28 point release, I'd be happy to help go through the SRU process to include such a change :-) (ping me on IRC if you want to discuss anything).

Olivier Tilloy (osomon) wrote :

Great, thanks Andrew for the detailed heads-up!
That sounds like a good SRU candidate if it lands in a 3.28 point release indeed. Can you point me to the upstream branch that has those changes?

Andrew Hayzen (ahayzen) wrote :

@osomon, sure :-)

This was the original PR against master https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/48 but i believe that has been superseded by https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/58 for the stable series 3.28. (jrocha from Endless was working on the code, he hangs around in #gnome-software on GIMPnet/GNOME IRC - i'm sure he'd be happy to answer any questions/help).

As as noted in the earlier comments, this is the bug that is tracking it upstream https://gitlab.gnome.org/GNOME/gnome-software/issues/235

AsciiWolf (asciiwolf) wrote :

The question is if the MR for gnome-3-28 will be ever merged. If you want Flatpak support in GS to be working properly on Ubuntu 18.04, please show your support and let Richard Hughes (or another GS dev) know. Either via comment in the MR (https://gitlab.gnome.org/GNOME/gnome-software/merge_requests/58) or by contacting the dev directly. Thanks!

The code from MR !58 should be working without any issues and is already used in Endless OS.

AsciiWolf (asciiwolf) wrote :

The other possible solution is to create a downstream patch for Ubuntu 18.04 using the code from MR !58.

AsciiWolf (asciiwolf) wrote :

So the MR !58 got closed and upstream doesn't seem to be interested in maintaining the Flatpak code in the gnome-3-28 branch anymore. They are focusing on the new code that is using the FlatpakTransaction API (and most likely won't be backported into the older GS branch) instead.

AsciiWolf (asciiwolf) wrote :

Here's a patch from the MR commits that can be applied downstream and fixes this issue.

AsciiWolf (asciiwolf) wrote :

@ahayzen Are you interested in doing a SRU for the Ubuntu 18.04 GS with this patch applied?

Andrew Hayzen (ahayzen) wrote :

@Asciiwolf, Personally I would prefer such a large change (229 additions and 37 deletions across 28 files) to be approved by upstream - especially as this is going into a LTS release. As it is not always clear what other behaviour this may change or break.

I understand that this is the patch that Endless are using in their distro, but I would need to understand how the flatpak plugin works before I would be able to make a decision.

Ultimately to get this change into Ubuntu it would need to be approved by Robert Ancell, so maybe Robert is the one who can advise if we should require upstream approval or if it is OK to distro patch. As this is quite an important fix for trying to use flatpak with a GUI in bionic.

I am happy to help with the SRU process and submitting the patches to the Ubuntu gnome-software git etc, I just feel I don't have enough knowledge of gnome-software's code to state if this is an OK change to carry.

(... or maybe we just need to wait until gnome-software is a flatpak/snap/appimage ;-) then we can magically get 3.30 :-) )

Andrew Hayzen (ahayzen) wrote :

(Also if this does get submitted I have a test plan for flatpak itself that includes UI tests for gnome-software, so this would help in doing the QA part)

AsciiWolf (asciiwolf) wrote :

@ahayzen I fully agree, let's wait for Robert's opinion about this. Also, please ignore the previous patch, the commits are in wrong order and even if I fix the order, it fails to apply for some reason. I will send the individual patches.

AsciiWolf (asciiwolf) wrote :
AsciiWolf (asciiwolf) wrote :

Here are the correct patches that apply without any issues on the latest gnome-3-28 branch.

The attachment "gs-add-related-refs-support.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
AsciiWolf (asciiwolf) on 2018-09-29
summary: - Flatpak runtime extensions are not installed when using GNOME Software
- on Ubuntu
+ Flatpak related refs are not installed when using GNOME Software on
+ Ubuntu 18.04
AsciiWolf (asciiwolf) wrote :

Any progress?

AsciiWolf (asciiwolf) wrote :

By the way, GS developer told me that they didn't want to merge it because they don't have enough people and time to further maintain it. So if someone from Canonical would be willing to maintain it in the gnome-3-28 branch, there should not be any problem having the patches upstream.

Robert Ancell (robert-ancell) wrote :

@AsciiWolf - if you want to propose this in 18.04 the best way is to do a MR to the ubuntu-3-28 branch on https://gitlab.gnome.org/Community/Ubuntu/gnome-software - that's where we get the patches for the .deb.

A brief look at the changes seems like it should be fine. I'd drop the test changes unless they break the build otherwise they'll make the patch enormous.

Changed in gnome-software (Ubuntu):
status: Confirmed → Fix Committed
Andrew Hayzen (ahayzen) wrote :

I believe this issue is resolved in cosmic and is only an issue in bionic. Normally such issues are marked as "Fix Released" for "ubuntu" and then proposed to the series bionic. I don't have the permissions to do this, Robert do you normally work with this workflow ?

Also I will test the normal flatpak manual test plan (which includes gnome-software UI tests) against this change once it is in -proposed to confirm this doesn't regress anything. And I am happy to help with any parts of the SRU process :-)

Jeremy Bicha (jbicha) on 2018-10-16
Changed in gnome-software (Ubuntu Bionic):
importance: Undecided → Low
status: New → Triaged
Changed in gnome-software (Ubuntu):
status: Fix Committed → Fix Released
Changed in gnome-software (Ubuntu Bionic):
assignee: nobody → Andrew Hayzen (ahayzen)
Andrew Hayzen (ahayzen) wrote :

@Robert, will you be generating a debdiff for the gnome-3-28 branch when it's ready? (I see you are making other changes and I'm not familiar how you are taking the git repo and turning the changes into debian patches).

Also I see that 3.28.1-0ubuntu4.18.04.4 [0] is currently stuck [1] in bionic-proposed as bug 1756379 has not been verified. I guess that this should be done before pushing the next release through :-)

0 - https://launchpad.net/ubuntu/+source/gnome-software/3.28.1-0ubuntu4.18.04.4
1 - http://people.canonical.com/~ubuntu-archive/pending-sru.html

Robert Ancell (robert-ancell) wrote :

@Andrew

The debian patches come from the ubuntu-3-28 branch [1] and are committed to the packaging branch [2]. Yeah, it's a bit complicated...

[1] https://gitlab.gnome.org/Community/Ubuntu/gnome-software/tree/ubuntu-3-28
[2] https://code.launchpad.net/~ubuntu-desktop/ubuntu/+source/gnome-software/+git/gnome-software/+ref/ubuntu/bionic

Hello AsciiWolf, or anyone else affected,

Accepted gnome-software into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gnome-software/3.28.1-0ubuntu4.18.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in gnome-software (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Andrew Hayzen (ahayzen) wrote :

The good news is that this now correctly installs themes when installing an app.

Unfortunately for me test "2.4 Update an app in GNOME Software" from the flatpak test plan [0], fails for me with the package from proposed.

After reverting to an older commit hash, then refreshing gnome-software it shows the update. But when I click "Update" the UI shows progress, hides the item and then it reappears again. Furthermore using the CLI you can see that the package hasn't been updated. So this change appears to be breaking an update.

When running from the console I get "01:38:32:0760 Gs tried overwriting org.gnome.Recipes.desktop key flatpak::Commit from 6d5f7c35ff1f3fd449b2789324843a89140257bb3cdca468f83a65162b177fa0 to 1ac37481845c65857a64e8c0680bae137b0b56faaf6f11f3f0d415a3205bc1c2" and if i run in verbose mode I get this https://pastebin.ubuntu.com/p/r8pWhpmcG7/

Note flipping back to 3.28.1-0ubuntu4.18.04.4 it works again.

Therefore I am going to have to mark this as verification-failed-bionic for now.

$ apt-cache policy gnome-software-plugin-flatpak
gnome-software-plugin-flatpak:
  Installed: 3.28.1-0ubuntu4.18.04.5
  Candidate: 3.28.1-0ubuntu4.18.04.5
  Version table:
 *** 3.28.1-0ubuntu4.18.04.5 500
        500 http://gb.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     3.28.1-0ubuntu4.18.04.4 500
        500 http://gb.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages
     3.28.1-0ubuntu4 500
        500 http://gb.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages

0 - https://wiki.ubuntu.com/Process/Merges/TestPlan/flatpak

tags: added: verification-failed-bionic
removed: verification-needed-bionic
AsciiWolf (asciiwolf) wrote :

You're right. I can confirm that the runtime related refs are now installed without any issue, however the new code seems to have problems with downgrades. :-( Unfortunately, I am not a good in C so I cannot fix it myself and Joaquim Rocha doesn't work on GNOME Software anymore.

Other possible solution to the related refs issue is to switch to the new GNOME Software 3.30.* version in Bionic instead of using an older one. This solution is recommended by GS devs because the new GS has many fixes (not only the Flatpak ones).

AsciiWolf (asciiwolf) wrote :

It would be great if some GS dev from Canonical could fix the current patches.

AsciiWolf (asciiwolf) wrote :

Proper Flatpak support is really important to end-users as Flatpak is becoming a standard in cross-distro application distribution.

Robert Ancell (robert-ancell) wrote :

I've re-uploaded the SRU with these changes removed. If someone can re-propose fixed patches and/or additional patches to resolve this then we can consider it for another SRU.

Changed in gnome-software (Ubuntu Bionic):
status: Fix Committed → Triaged
Chris Halse Rogers (raof) wrote :

Hello AsciiWolf, or anyone else affected,

Accepted gnome-software into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gnome-software/3.28.1-0ubuntu4.18.04.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in gnome-software (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic
removed: verification-failed-bionic
Changed in gnome-software (Ubuntu Bionic):
status: Fix Committed → Triaged
tags: removed: verification-needed verification-needed-bionic
Chris Halse Rogers (raof) wrote :

Marking as verification-done; the change that regressed has been reverted, so this bug isn't actually fixed, but it also hasn't introduced the downgrade problem.

tags: added: verification-done
tags: added: verification-done-bionic
removed: verification-done
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package gnome-software - 3.28.1-0ubuntu4.18.04.8

---------------
gnome-software (3.28.1-0ubuntu4.18.04.8) bionic; urgency=medium

  * Re-released to generate correct .changes file

gnome-software (3.28.1-0ubuntu4.18.04.7) bionic; urgency=medium

  * debian/rules:
  * debian/patches/0001-packagekit-Add-configure-option-to-use-autoremove-fe.patch:
    - Build with PackageKit autoremove support (LP: #1552792)
  * debian/control:
  * debian/patches/0023-snap-Don-t-treat-auth-cancellation-as-an-error.patch:
    - Stop cancelling snapd authorization triggers error notification
      (LP: #1785240)

gnome-software (3.28.1-0ubuntu4.18.04.6) bionic; urgency=medium

  * debian/patches/0023-Revert-Revert-flatpak-Use-list-of-related-apps-for-i.patch:
  * debian/patches/0024-flatpak-Check-if-a-related-app-is-installed-even-whe.patch:
  * debian/patches/0025-flatpak-Include-related-refs-to-the-app-s-runtime-on.patch:
    - Disable the Flatpak changes - SRU testing showed they broke downgrades

gnome-software (3.28.1-0ubuntu4.18.04.5) bionic; urgency=medium

  * debian/patches/0023-Revert-Revert-flatpak-Use-list-of-related-apps-for-i.patch:
  * debian/patches/0024-flatpak-Check-if-a-related-app-is-installed-even-whe.patch:
  * debian/patches/0025-flatpak-Include-related-refs-to-the-app-s-runtime-on.patch:
    - Pull related flatpak refs (LP: #1754864)
  * debian/patches/0003-Sort-snaps-before-other-apps.patch:
    - Fix snap search result ordering (LP: #1798228)
  * debian/patches/0001-Set-reboot-notifications-as-urgent-to-avoid-them-tim.patch:
    - Stop reboot notification from timing out (LP: #1719797)
  * debian/patches/0001-lib-Allow-opening-CAB-files-that-resolve-more-than-o.patch:
    - Support composite CAB files (LP: #1798470)

 -- Robert Ancell <email address hidden> Wed, 21 Nov 2018 12:29:27 +1300

Changed in gnome-software (Ubuntu Bionic):
status: Triaged → Fix Released

The verification of the Stable Release Update for gnome-software has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

AsciiWolf (asciiwolf) wrote :

Was this issue really fixed? I thought the patches were reverted.

Jeremy Bicha (jbicha) on 2018-12-11
Changed in gnome-software (Ubuntu Bionic):
status: Fix Released → Triaged
tags: removed: verification-done-bionic
To post a comment you must log in.