Firefox snap crashed after snapd snap was refreshed in the background

[Olivier Tilloy]

I was using the firefox snap for regular browsing when it crashed (the crash reporter dialog popped up). Out of curiosity I ran `snap changes`, and I observed that the snapd snap had just been refreshed a minute before the crash happened.

I'm attaching the output of `snap changes`, `snap tasks $TASKID` (corresponding to the snapd refresh) and the relevant portion of `journalctl --no-pager -b0` (from just before the refresh happened to after firefox was restarted after crashing).

[Olivier Tilloy (osomon)]

A crash of the firefox snap I've been observing a few times recently is https://bugzilla.mozilla.org/show_bug.cgi?id=1732718 (where the crash is caused by the use of a custom cursor theme that isn't available in gtk-common-themes). I wonder if refreshing the snapd snap might be triggering that same crash, which would explain why others haven't been able to reproduce the problem?

[Olivier Tilloy (osomon)]

Unfortunately, about:crashes doesn't list any crash for today, it looks like the crash wasn't properly recorded.

[Ian Johnson (anonymouse67)]

FWIW, I can't reproduce the crash either with the cursor theme switch, but I am in an X11 session. I will test switching cursor themes in a Wayland session to match your bug report.

[Olivier Tilloy (osomon)]

I must also note I had other snaps running when the snapd refresh happened (multipass, lxd, snap-store, telegram-desktop), and they didn't seem to be negatively affected.

[Ian Johnson (anonymouse67)]

I also tried with the firefox snap on 21.10 with Wayland and couldn't reproduce a crash refreshing snapd from stable -> edge and back again.

I can however easily reproduce the same crash after changing the cursor theme to DMZ White on my 21.10 Wayland session, so that bug is for sure happening. @osomon, which cursor theme did you have configured when you saw the firefox crash?

[Ian Johnson (anonymouse67)]

FWIW I tried both with the default Yaru cursor and with the DMZ White cursor and couldn't reproduce a crash if I set the cursor theme with gnome-tweaks, launched the firefox snap, then refreshed snapd stable -> edge -> stable again.

[Olivier Tilloy (osomon)]

I'm using the Oxy-blue theme (from the oxygen-cursor-theme package).

[Sebastien Bacher (seb128)]

I just saw the issue on an hirsute/XFCE/xorg machine where thunderbird and firefox snaps where is use, they crashed out of nowhere, checking 'snap changes' there was a snap refresh around the time of the issue. The cursor in use is Yaru

[shao.lo (shao-lo)]

I've used firefox forever with very few problems. When I updated to the last Ubuntu it installed the snap version. Ever since then I get periodic crashes. Looking at the comments in this bug today I saw the same thing
snap changes
ID Status Spawn Ready Summary
400 Done today at 11:13 CST today at 11:14 CST Auto-refresh snap "firefox"
A few minutes later I got a crash when I switched to firefox to read some email.
About:crashes confirms the crash report time after the snap time. I'll check this when it happens again, but very annoying.

[Olivier Tilloy (osomon)]

@shao.lo: to mitigate the problem, I suggest enabling Refresh App Awareness¹, which should prevent snapd from refreshing snaps while they are running.

¹ https://forum.snapcraft.io/t/wip-refresh-app-awareness/10736

[Gerhard Aigner (gerhard-aigner)]

I've been an ubuntu user since ~2009 and have never seen probems with firefox until the snap version came into play. It crashes randomly and I verified that the crashes do not always correspond to an update of firefox or one of its dependencies...

[Lars Ljung (larslj)]

Today's update to 97.0 did not crash. Instead it shut down Firefox. I'm not sure if this is much better. After a manual restart I got the option to restore the previous session.

[Olivier Tilloy (osomon)]

What do you mean by "it shut down Firefox"? Did it warn you that it was going to close the application? Because if it didn't, that's most likely a crash (even if the crash reporter dialog didn't show up afterwards for whatever reason).

[Lars Ljung (larslj)]

This happened while I was away. Previously Firefox would still be running when I came back. As soon as I tried to do something it would show me the crash reporter dialog. But I guess this change of behavior could be random. So far the update has never occurred when I was actively using Firefox.

[Olivier Tilloy (osomon)]

Lars, if you open about:crashes, is there an entry with a timestamp that would match what you saw yesterday?

[shao.lo (shao-lo)]

@osomon: Thanks. After using 'snap set core experimental.refresh-app-awareness=true' I did get a message this morning that snap was updating and that I should restart firefox in the next 14 days (or something like that). Hopefully that is the end of my mystery crashes. Thanks again!!

[Lars Ljung (larslj)]

Yes, I do have an entry in about:crashes from 2022-02-14. So it seems it crashed.

However, I'm now writing this from Firefox 97.0, after 97.0.1 was installed some hours ago. So it did not crash this time. But there were no notification either. The only visible sign of the update is that the Firefox icon on the dock no longer has the small orange dot next to it.

[Matthew Ruffell (mruffell)]

I have had a pretty bad run of Firefox refreshing when I turn my computer on in the mornings, and I think this needs to be prioritized for being fixed before Jammy is released.

I know there is talk on the discourse thread about enabling refresh-app-awareness:

$ sudo snap set core experimental.refresh-app-awareness=true

Will this graduate out of experimental? Will this get fixed in snapd core?

[Thibauld (thibauld)]

I see this bug still has "New" status and importance "Undecided" so I have to weight in :)
The Firefox is nothing less than completely broken on Ubuntu 2.10 (it was a fresh install).
Firefox crashes at least once a day (most of the time in the morning, a few minutes after I sit down in front of my computer, after a few minutes of FF use).

The only good thing about this bug is that I can only applaud the robustness of FF's restore session feature, which really works well :)

Other than that, this bug is definitely "confirmed" and, regarding its importance, I don't see how the next LTS could be called "stable" with such a broken FF experience.

Happy to provide more info if needed. I just enabled this today:
sudo snap set core experimental.refresh-app-awareness=true

Hopefully, it will reduce the crash frequency.

[Olivier Tilloy (osomon)]

This upstream bug is also relevant, although it doesn't happen only when snapd is refreshed, but every time the firefox snap is refreshed too: https://bugzilla.mozilla.org/show_bug.cgi?id=1737810.

[Matthew Ruffell (mruffell)]

I was reading my email this morning, right after I had turned my computer on, and I got this notification:

Pending update of "firefox" snap
Close the app to avoid disruptions (14 days left)

And you know what? Firefox did not crash!

This is marvelous! Exactly what I was looking for. Looks like refresh-awareness is what we all need. I finished reading my email, firefox still stable, closed it on my own time, and let it refresh.

Ian, can this feature please be prioritised to move from experimental to snapd core, and can we enable it by default for Jammy? There are going to be a lot of users moving from deb firefox on Focal to snap firefox on Jammy, and these regular snap refreshes are going to impact a lot of people.

I do agree that refresh-awareness is the way to go.

[Thibauld (thibauld)]

Quick update after a few days of using

`$ sudo snap set core experimental.refresh-app-awareness=true`

No more crashes (same behavior as @mruffell wrote above), which is great!

Few annoying things remaining from what I could witness as a user:

1. I thought that closing and restarting Firefox would take care of updating the snap. Nope, turns out I had to manually do a `sudo snap refresh`
2. With Chromium, I had to manually kill all chromium processes before `sudo snap refresh` would work because, somehow, there was still snap chromium processes running after closing Chromium.

[Ian Johnson (anonymouse67)]

To be clear, there are two "fixes" here:

1. My fix to the underlying mount namespace code which is immediately causing a crash in firefox whenever a content snap or snapd itself is refreshed. That is under review here: https://github.com/snapcore/snapd/pull/10676 and is due to be released for snapd 2.55 (we will not release 2.55 without this fix).

2. The more general fix to prevent refreshes of snaps in general when they are being used is refresh app awareness and that is orthogonal to the fix I mentioned above. Pawel has posted the final call for testing of the feature at https://forum.snapcraft.io/t/refresh-app-awareness-call-for-testing/29123 so providing positive feedback that the feature works in preventing issues there would be appreciated as well as other issues or questions about the feature. I don't consider this specific launchpad bug to be about enabling refresh app awareness FWIW, I believe we have another bug "tracking" that, but regardless, enabling it by default is very high on our priority for 22.04.

Thanks,
Ian

[Ian Johnson (anonymouse67)]

The first fix about fixing the mount namespace issue has landed and is in 2.55, which is now on the beta channel. If folks could confirm that the issue goes away for them after refreshing to snapd 2.55 with:

snap refresh snapd --channel=beta

After testing you can switch back to tracking the stable channel of snapd with:

snap refresh snapd --channel=stable

Thanks

[Ian Johnson (anonymouse67)]

Note that you may have to close out all instances of a snap like firefox even after refreshing snapd for the fix to take effect. A reboot will also do the trick.

[Olivier Tilloy (osomon)]

That's excellent news, thanks Ian, Alberto and everyone else involved.

[Bruno (brunoaiss)]

Problem still present. I still can't increase days from the 13 to a comfortable value for my system (infinite) and then just crashes the program. I've had data loss because of this!

Additionally, it keeps nagging me repetitively throughout the day to update when I'm trying to focus on other things.

Even worse: it wakes up my PC screen with the notification which I don't care. Just wait for the program to close to update it! The computer will be restarted at some point in time. That program will be restarted then.

The solution is easy to implement (assuming enough storage is available):

Download the .snap at snapd's convenience and leave it there waiting for the program to close. Then, just swap the snap in use as soon as the program stops.
All the hard work for that has already been done. What am I missing here?

A notification the 13 days (not tries) is fine. However, a notification before 3 days (debatable) passed is already too much!

[Olivier Tilloy (osomon)]

Bruno, what versions of snapd and firefox are installed on your system?