User notification is missing when firmware updates from fwupd are available

[Crag Wang]

There is no user notification poped up when firmware updates are available.

Steps to reproduce:
1. Install Ubuntu 22.04
2. After OOBE, launch the software center and switch to "Updates" tab
3. Several firmware updates are list but no user notification at all
   (from within the upper middle area, along the calendar/time)

Expected result:
- user notification is seen to remind users for an available update

Hardware: Dell XPS 13 9380
OS: Ubuntu 22.04.1
Software Center: snap-store 41.3-64-g512c0ff, rev-599

[Yuan-Chen Cheng (ycheng-twn)]

Per double check, the bug that prevent fwupd to refresh should be lp:1969976

Per check, the failure log in syslog shall be something like:

Sep 29 20:02:04 XXXX systemd[1]: Starting Refresh fwupd metadata and update motd...
Sep 29 20:02:04 XXXX systemd[1]: fwupd-refresh.service: Main process exited, code=exited, status=1/FAILURE
Sep 29 20:02:04 XXXX systemd[1]: fwupd-refresh.service: Failed with result 'exit-code'.
Sep 29 20:02:04 XXXX systemd[1]: Failed to start Refresh fwupd metadata and update motd.

Given it can't properly refresh the meta-data, then there won't be a notification, just in case that there are some other issue block notification

AI: install the updated fwupd from ppa and test again.

[Yuan-Chen Cheng (ycheng-twn)]

Install fwupd from https://launchpad.net/~ycheng-twn/+archive/ubuntu/fwupd-jammy-6
Then run "systemctl start fwupd-refresh.service" to force refresh fwupd meta data.

Wait for some more time to see if we can get a notification.

However I found one thing, there once have a process running in the background.

/snap/snap-store/599/usr/bin/snap-store --gapplication-service

However, after removing and re-install the snap-store snap, I can't see that's running. Not
sure if something is wrong.

[Yuan-Chen Cheng (ycheng-twn)]

Also found that on snap-store version: 41.3-64-g512c0ff rev: 599 on jammy, can't find the button for configuration (on the right-upper corner) Is that expected?

[Yuan-Chen Cheng (ycheng-twn)]

If we run gnome-software deb, we have the configuration button and function. Check attachments.

[Heather Ellsworth (hellsworth)]

fwupd isn't refreshing and there is already a bug for that. Then the issues left are:

1. snap-store isn't autostarting. I looked into this more and noticed that the snap-store snap on 20.04 (focal) wouldn't start on a fresh install, but once I started it, it had no problem autostarting on reboot. On 22.04.1 and 22.10, even if you start the snap-store on a fresh install, it still doesn't autostart on reboot.

2. The missing config menu is I think a real bug that has not already been filed, so I've created:
https://bugs.launchpad.net/snap-store-desktop/+bug/1992528

[Heather Ellsworth (hellsworth)]

I just learned that upstream gnome-software removed this, so it is not a bug.

[Crag Wang (crag-wang)]

@Heather Ellsworth,

Would user notification be functional for available software/firmware updates if snap-store got started successfully?

[Heather Ellsworth (hellsworth)]

I thought there was a problem with autostart but there isn't. It turns out a user needs to launch the snap-store manually once in order for it to be autostarted (which makes sense).

The problem is that the updates available in the snap-store Updates tab isn't populating the notifications. One reason the gnome-software deb could be working and the snap-store isn't, is that the versions are different:
- gnome-software deb is at version 43.0
- snap-store snap is at version 41.3

I've found some commits in the upstream gnome-software 43.0 that fiddle with how notifications are done (that came after 41.3):

commit 64585205437371c459d5de792d50af95e23ba346
Author: Milan Crha <email address hidden>
Date: Wed Jun 15 18:19:36 2022 +0200

    update-monitor: Change when notifications are shown

    This handles the fact that offline updates can be installed either
    all in once or none of them, which is reflected in the notification.

commit 1aaa9617fc81651f17ed2f864bec35b958d67eed
Author: Milan Crha <email address hidden>
Date: Wed Jun 15 18:17:18 2022 +0200

    update-monitor: Force reload of the Updates page before showing notification

    Thus it reflects what the update-monitor notifies about.

So I wonder if the snap-store snap would fix the notifications by simply updating its gnome-software base.

[Sergio Costas (rastersoft-gmail)]

The autostart in the snap store is tricky, in the sense that not only the user has to launch the application once, but also that it is set only for "ubuntu-software" app, not for "snap-store", so launching the later seems to not enable the autostart, only launching the former.

[Sebastien Bacher (seb128)]

Ok, so after some activity there it's a bit unclear what's the initial issue is reported exactly

@oem how do you test that exactly? could you check if there is a snap-store service active on the machine having the issue?

if it's not, did you ever user the snap-store UI before and how did you start it?

One problem we have is that snap-store hasn't been manually started the service isn't going to be registered and users will never see the notification.

Then we have the issue of user starting 'snap-store' isn't going to register the autostart as 'ubuntu-software' would, there is a fix for that on https://gitlab.gnome.org/Community/Ubuntu/gnome-software/-/merge_requests/60/

[Yuan-Chen Cheng (ycheng-twn)]

@Seb, the requirement from the end-user is straightforward,
end-user needs a way to get a notification if there are
fwupd/firmware update available and then they could
just click the button to run the firmware upgrade.

Given currently snap-store is the only default-installed
app that could do this, so the user experience needed is:

After installation, as snap-store will be pre-seed, and
gnome-software won't be installed, then snap-store needs
to be able to regular check if there are any pending
installation of fwupd/firmware, and notify user
about it.

Given so, does the current snap-store support
such user requirements? If not, it's just a bug-fix,
or does it need more re-design (if it can't just behave as
update-manager that will always up and run regularly)

[Sebastien Bacher (seb128)]

@Yuan-Chen, the user experience you describe makes sense yes and I agree that's how it should work. I'm just trying to make sure you correctly identify the current issues which are on the way of getting that experience.

There is a problem we need to solve that the snap will only act as a service if it was manually started once. We need to work on figuring out how we resolve that.

But I'm also interested to know if there is more to solve once that problem will be resolved and I would prefer to not have to wait for the fix to land to figure if there is more because it will create more delays

[Crag Wang (crag-wang)]

The expected user notification should be similiar to:
$ notify-send -i info "New firmware updates are available"

I've installed fwupd 1.7.9 to address fwupd-refresh/systemd bug[1] as well as the snap-store[2] from comment #15 specifically from the edge channel. I don't see user notification pops up yet, maybe additional work is required according to comment 13?

[1] https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1969976.
[2] 41.3-65-g234ecdf

--
$ snap list
Name Version Rev Tracking Publisher Notes
core20 20220826 1623 latest/stable canonical✓ base
snap-store 41.3-65-g234ecdf 634 latest/edge canonical✓ -

$ dpkg -l | grep fwupd
ii fwupd 1.7.9-1~22.04.1

[Yuan-Chen Cheng (ycheng-twn)]

@Seb about manually starting the service, could you confirm that:

1. For now, we need to run the snap-store once every reboot, right?
2. To check if it's running, it can be checked by finding if the process
   /snap/snap-store/599/usr/bin/snap-store is running under the login
   user name, is that right?

[Sebastien Bacher (seb128)]

I've poked a bit to it today without much luck. I tried to see if gnome-software was behaving better for me but it doesn't display notifications either

I've reported a bug upstream https://gitlab.gnome.org/GNOME/gnome-software/-/issues/1943 which includes some debugging from my system and code pointers from upstream.

The code is not as trivial as 'should display notification on start', it's meant to display them once a week (timestamp stored in gsettings 'org.gnome.software update-notification-timestamp') unless they are urgent updates (which firmware don't seem to be).

[Heather Ellsworth (hellsworth)]

@ycheng-twn To answer your two questions:

> 1. For now, we need to run the snap-store once every reboot, right?

The user should only need to start the snap-store manually one time at all, not every reboot. This gets autostart going so the snap store is autostarted on reboots.

> 2. To check if it's running, it can be checked by finding if the process
   /snap/snap-store/599/usr/bin/snap-store is running under the login
   user name, is that right?

Yes, or whatever version (instead of 599). The autostart had an issue afterall mentioned in comment #14 and it has been fixed in the snap-store build (rev 634) in --edge.

We still have a problem getting notifications at all from the snap-store.

[Robert Ancell (robert-ancell)]

The following DBus trace shows that snap store requesting a notification:

method call time=1666146527.796748 sender=:1.79 -> destination=:1.91 serial=2298 path=/org/freedesktop/portal/desktop; interface=org.freedesktop.impl.portal.Notification; member=AddNotification
   string "snap.snap-store"
   string "updates-available"
   array [
      dict entry(
         string "title"
         variant string "Software Updates Ready to Install"
      )
      dict entry(
         string "body"
         variant string "Software updates are waiting and ready to be installed."
      )
      dict entry(
         string "priority"
         variant string "normal"
      )
      dict entry(
         string "default-action"
         variant string "app.set-mode"
      )
      dict entry(
         string "default-action-target"
         variant string "updates"
      )
   ]
method call time=1666146527.797210 sender=:1.91 -> destination=:1.38 serial=323 path=/org/gtk/Notifications; interface=org.gtk.Notifications; member=AddNotification
   string "snap.snap-store"
   string "updates-available"
   array [
      dict entry(
         string "title"
         variant string "Software Updates Ready to Install"
      )
      dict entry(
         string "body"
         variant string "Software updates are waiting and ready to be installed."
      )
      dict entry(
         string "priority"
         variant string "normal"
      )
      dict entry(
         string "default-action"
         variant string "app.set-mode"
      )
      dict entry(
         string "default-action-target"
         variant string "updates"
      )
   ]
method return time=1666146527.797420 sender=:1.91 -> destination=:1.79 serial=324 reply_serial=2298
error time=1666146527.798605 sender=:1.38 -> destination=:1.91 error_name=org.gtk.Notifications.InvalidApp reply_serial=323
   string "The app by ID "snap.snap-store" could not be found"

It is being rejected in GNOME Shell (js/ui/notificationDaemon.js):

        try {
            source = this._ensureAppSource(appId);
        } catch (e) {
            if (e instanceof InvalidAppError) {
                invocation.return_dbus_error('org.gtk.Notifications.InvalidApp',
                    `The app by ID "${appId}" could not be found`);
                return;
            }
            throw e;
        }

[Robert Ancell (robert-ancell)]

jamesh helped here - if we run with this env variable we use the old notification system and they show:
GNOTIFICATION_BACKEND=freedesktop snap-store

I suggest we add that to the snap package.

[James Henstridge (jamesh)]

This can probably be solved by setting GNOTIFICATION_BACKEND=freedesktop.

The problem is caused when the xdg-desktop-portal Notifications API proxies to gnome-shell's new org.gtk.Notifications API rather than the old org.freedesktop.Notifications API. It uses the package ID as the application ID in these calls. In this particular case, the x-d-p package ID would be "snap.snap-store".

Gnome-shell rejects this application ID because there is no "snap.snap-store.desktop" file in ${XDG_DATA_DIRS}/applications to associate the notification with. Unfortunately it isn't as simple as updating x-d-p to send the snap's desktop ID instead ("snap-store_snap-store.desktop" in this case), since the the GTK notifications API requires that the application ID be a D-Bus activatable desktop ID, and "snap-store_snap-store" is not a valid D-Bus name. We'd also need to enable user daemons in snapd before we could even be D-Bus activatable.

Setting GNOTIFICATION_BACKEND will cause the GApplication notification code not to use the portal and instead call the FDO notifications API directly, which is currently allowed by the "desktop" interface. It doesn't fix the underlying problem, but lets us avoid it.

[Sebastien Bacher (seb128)]

The snap has been updated to set GNOTIFICATION_BACKEND=freedesktop in its env now, the fix is in the edge channel now

snap-store with only display notifications once a week for non important update and once a day for important ones (but currently firmware updates are not labelled as important, another fix is coming for that which I'm currently testing and will merge propose later)

doing

$ for key in `gsettings list-keys org.gnome.software | grep timestamp`; do echo $key was `gsettings get org.gnome.software $key`; gsettings reset org.gnome.software $key; done

then restarting snap-store correctly display the notifications now (either the UI or using --gapplication-service)

we still have the issue than the store needs to be started manually once for the autostart to be registered and for users to eventually get notified

[Sebastien Bacher (seb128)]

the fixed version has been promoted to the candidate channel now

[Yuan-Chen Cheng (ycheng-twn)]

@Crag, could you comment if the snap in the candidate work as expected,
and if any further requests?

[Crag Wang (crag-wang)]

I'm seeing an error (no such schema) until I have gnome-software installed in my system.

$ gsettings list-keys org.gnome.software
No such schema “org.gnome.software”

After gnome-software is installed, and resetted the keys in the schema. I started snap-store but I don't see available software updates in the tab nor the user notification. On the attached is the dbus-session log upon snap-store was launched.

--
Name Version Rev Tracking Publisher Notes
bare 1.0 5 latest/stable canonical✓ base
core 16-2.57.2 13886 latest/stable canonical✓ core
core18 20220831 2566 latest/stable canonical✓ base
core20 20220826 1623 latest/stable canonical✓ base
gnome-3-38-2004 0+git.6f39565 119 latest/stable canonical✓ -
gtk-common-themes 0.1-81-g442e511 1535 latest/stable/… canonical✓ -
snap-store 41.3-66-gfe1e325 638 latest/candidate canonical✓ -
snapcraft 7.1.4 8210 latest/stable/… canonical✓ classic
snapd 2.57.4 17336 latest/stable canonical✓ snapd

||/ Name Version Architecture Description
+++-==============-=============-============-=================================
ii gnome-software 41.5-2ubuntu2 amd64 Software Center for GNOME

[Crag Wang (crag-wang)]

On a Ubutnu 22.04 fresh installed system, I can see the application list in snap-store's Updates tab where the package version is 41.3-60-gfe4703a. After upgraded to 41.3-66-gfe1e325 the application list is somewhat missing, only firmware updates are listed.

At a point, an user notification pops up and saying "Software Ready". Clicked the notification brings me to the snap-store, it's hard for me to distinguish who the caller is sending the notification between the software packages and firmware/fwupd, can someone please eleborate more the details?

Ubuntu 22.04 new installed (app list is available in the Updates tab):
  Name Version Rev Tracking Publisher Notes
  snap-store 41.3-60-gfe4703a 582 latest/stable/… canonical✓ -

After snap refresh (app list become missing in the Updates tab):
  Name Version Rev Tracking Publisher Notes
  snap-store 41.3-66-gfe1e325 638 latest/candidate canonical✓ -

[Heather Ellsworth (hellsworth)]

I just did a fresh 22.04 install (without updates) and had snap-store 41.3-60-gfe4703a installed. Then I refreshed my snap-store to 41.3-66-gfe1e325 (to track what is in candidate). With both versions of the snap-store, my Updates tab shows only snaps listed. (Since my testing is on a VM, there will not be firmware updates available.) I don't see a visible difference between these two versions, like @Crag does, but I can try a hardware install to see if this changes things.

[Crag Wang (crag-wang)]

Today I saw another user notification titled as "Software Updates Ready to Install" other than previously "Software Ready", how to distinguish and identify the caller of these emits?

That'd be super excited if we can confirm the user notification comes from the fwupd plugin, and so as to close this issue confidently. Please help.

[Sebastien Bacher (seb128)]

You can try to see what
$ snap-store --verbose
is saying

But one way to also check is to clear any pending snap update, then reset the gsettings keys as described in #25 and restart snap-store. The only pending updates in the list should be firmware ones and if you get the notification then it's about the firmware update

[Crag Wang (crag-wang)]

Confirming that desktop notification for firmware updates is seen. The snap-store in use is 41.3-66-gfe1e325 while now it lives in the candidate channel.

@Sebastien Bacher, plan to move this version into stable?

The text of poped up notification in my case:
- title: Software Updates Ready to Install
- body : Software updates are waiting and ready to be installed.

Else case see https://git.launchpad.net/snap-store-desktop/tree/src/gs-update-monitor.c?h=41.3#n200

$ snap list snap-store
Name Version Rev Tracking Publisher Notes
snap-store 41.3-66-gfe1e325 638 latest/candidate canonical✓ -

$ snap-store --verbose
..(skipped)
07:52:01:0297 Gs running get-updates on plugin=snap with dedupe-flags=7 with refine-flags=require-update-details,require-update-severity with timeout=60 on apps */cabinet/*/com.dell.wd197fb217aa.firmware/*,*/cabinet/*/com.dell.wd19cd357cf1.firmware/*,*/cabinet/*/com.dell.uefia40a57ef.firmware/*,*/cabinet/*/org.linuxfoundation.dbx.x64.firmware/*, elapsed time since creation 1174ms
07:52:01:0297 Gs got 4 updates
07:52:01:0297 Gs Getting updates
07:52:01:0297 Gs Chaining cancellation from 0x55f3afa271a0 to 0x55f3aff9cda0
07:52:01:0297 Gs Disconnecting cancellable 0x55f3afa271a0
07:52:01:0297 Gs no refine flags set for transaction
07:52:01:0297 Gs running download on plugin=packagekit-refresh with dedupe-flags=7 with propagate-error=True with timeout=60 on apps */cabinet/*/com.dell.wd197fb217aa.firmware/*,*/cabinet/*/com.dell.wd19cd357cf1.firmware/*,*/cabinet/*/com.dell.uefia40a57ef.firmware/*,*/cabinet/*/org.linuxfoundation.dbx.x64.firmware/*
07:52:01:0297 Gs should_notify_about_pending_updates: last_test_days:365 n-apps:4 should_download:1 has_important:0 any_important_downloaded:0 all_downloaded:1 any_downloaded:1 res:1 reason:Software Updates Ready to Install|Software updates are waiting and ready to be installed.
07:52:01:0297 Gs Notify about update: 'Software Updates Ready to Install'

[Sebastien Bacher (seb128)]

The new revision has been moved to stable now, sorry again for the delay

[Crag Wang (crag-wang)]

Verified pass with snap-store 41.3-66-gfe1e325 from the stable channel, thanks.

$ snap list snap-store
Name Version Rev Tracking Publisher Notes
snap-store 41.3-66-gfe1e325 638 latest/stable canonical✓ -

[Crag Wang (crag-wang)]

re comment #25

> we still have the issue than the store needs to be started manually once for the autostart to be registered and for users to eventually get notified

Hi Sebastien Bacher,
Have you got a chance to release a fix for the auto-startup issue?

[Sebastien Bacher (seb128)]

No, we haven't worked on resolving the autostart issue. It only impacts people who never used the software store so probably lower importance but would be nice to resolve still yes