firmware-updater snap not installed during upgrade

[Brian Murray]

I had the version of snapd from lunar-proposed installed and it still failed to install the firmware-updater snap.

ProblemType: Bug
DistroRelease: Ubuntu 23.10
Package: ubuntu-release-upgrader-core 1:23.10.10
ProcVersionSignature: Ubuntu 6.5.0-10.10-generic 6.5.3
Uname: Linux 6.5.0-10-generic x86_64
ApportVersion: 2.27.0-0ubuntu5
Architecture: amd64
CasperMD5CheckMismatches: ./boot/grub/grub.cfg
CasperMD5CheckResult: fail
CrashDB: ubuntu
CurrentDesktop: ubuntu:GNOME
Date: Fri Oct 27 12:13:12 2023
InstallationDate: Installed on 2023-07-27 (92 days ago)
InstallationMedia: Ubuntu 23.04 "Lunar Lobster" - Daily amd64 (20230412)
PackageArchitecture: all
ProcEnviron:
 LANG=en_US.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=xterm-256color
 XDG_RUNTIME_DIR=<set>
SourcePackage: ubuntu-release-upgrader
Symptom: ubuntu-release-upgrader
UpgradeStatus: Upgraded to mantic on 2023-10-27 (0 days ago)
VarLogDistupgradeTermlog:

[Brian Murray (brian-murray)]

Relevant part of the log in a terminal:

Processing snap replacements

refreshing snap core22

installing snap firmware-updater
error: cannot perform the following tasks:
- Automatically connect eligible plugs and slots of snap "firmware-updater" (internal error: auto-connect of &{"firmware-updater:fwupd" "snapd:fwupd"} failed: snap "snapd" has no slot named "fwupd")

refreshing snap gnome-42-2204

refreshing snap gtk-common-themes

refreshing snap snap-store

refreshing snap snapd-desktop-integration

refreshing snap firefox

Searching for obsolete software

[Brian Murray (brian-murray)]

Here's the screenlog from when I ssh'ed in to a similar system and ran the dist-upgrade

[Steve Langasek (vorlon)]

The fwupd slot is available for me on a lunar container before upgrading.

What does 'snap interfaces | grep fwupd' show before you start the upgrade, and again after the error happens?

And after the upgrade, you can see the snap contents in /snap/snapd/current, right? We thought we had the snap.mount error fixed but maybe the mounts are still breaking again some other way

[Brian Murray (brian-murray)]

The output of some snap commands:

bdmurray@clean-lunar-amd64:~$ snap changes
ID Status Spawn Ready Summary
5 Done today at 13:48 PDT today at 13:48 PDT Switch "core22" snap to channel "stable"
6 Error today at 13:48 PDT today at 13:49 PDT Install "firmware-updater" snap from "stable/ubuntu-23.10" channel
7 Done today at 13:49 PDT today at 13:49 PDT Switch "gnome-42-2204" snap to channel "stable/ubuntu-23.10"
8 Done today at 13:49 PDT today at 13:49 PDT Switch "gtk-common-themes" snap to channel "stable/ubuntu-23.10"
9 Done today at 13:49 PDT today at 13:49 PDT Switch "snap-store" snap to channel "stable/ubuntu-23.10"
10 Done today at 13:49 PDT today at 13:49 PDT Switch "snapd-desktop-integration" snap to channel "stable/ubuntu-23.10"
11 Done today at 13:49 PDT today at 13:49 PDT Switch "firefox" snap to channel "stable/ubuntu-23.10"

bdmurray@clean-lunar-amd64:~$ snap tasks 6
Status Spawn Ready Summary
Done today at 13:48 PDT today at 13:49 PDT Ensure prerequisites for "firmware-updater" are available
Undone today at 13:48 PDT today at 13:48 PDT Download snap "firmware-updater" (109) from channel "stable/ubuntu-23.10"
Done today at 13:48 PDT today at 13:48 PDT Fetch and check assertions for snap "firmware-updater" (109)
Undone today at 13:48 PDT today at 13:48 PDT Mount snap "firmware-updater" (109)
Undone today at 13:48 PDT today at 13:48 PDT Copy snap "firmware-updater" data
Undone today at 13:48 PDT today at 13:48 PDT Setup snap "firmware-updater" (109) security profiles
Undone today at 13:48 PDT today at 13:48 PDT Make snap "firmware-updater" (109) available to the system
Error today at 13:48 PDT today at 13:48 PDT Automatically connect eligible plugs and slots of snap "firmware-updater"
Hold today at 13:48 PDT today at 13:48 PDT Set automatic aliases for snap "firmware-updater"
Hold today at 13:48 PDT today at 13:48 PDT Setup snap "firmware-updater" aliases
Hold today at 13:48 PDT today at 13:48 PDT Run install hook of "firmware-updater" snap if present
Hold today at 13:48 PDT today at 13:48 PDT Run default-configure hook of "firmware-updater" snap if present
Hold today at 13:48 PDT today at 13:48 PDT Start snap "firmware-updater" (109) services
Hold today at 13:48 PDT today at 13:48 PDT Run configure hook of "firmware-updater" snap if present
Hold today at 13:48 PDT today at 13:48 PDT Run health check of "firmware-updater" snap

......................................................................
Automatically connect eligible plugs and slots of snap "firmware-updater"

2023-10-27T13:48:58-07:00 ERROR internal error: auto-connect of &{"firmware-updater:fwupd" "snapd:fwupd"} failed: snap "snapd"
has no slot named "fwupd"

[Brian Murray (brian-murray)]

On Fri, Oct 27, 2023 at 11:01:38PM -0000, Steve Langasek wrote:
> The fwupd slot is available for me on a lunar container before
> upgrading.
>
> What does 'snap interfaces | grep fwupd' show before you start the
> upgrade, and again after the error happens?
>
> And after the upgrade, you can see the snap contents in
> /snap/snapd/current, right? We thought we had the snap.mount error
> fixed but maybe the mounts are still breaking again some other way

bdmurray@clean-lunar-amd64:~$ ls -alh /snap/snapd/current/
total 8.0K
drwxr-xr-x 2 root root 4.0K Aug 15 09:14 .
drwxr-xr-x 4 root root 4.0K Aug 15 09:14 ..

--
Brian Murray

[Brian Murray (brian-murray)]

bdmurray@clean-lunar-amd64:~$ apt-cache policy snapd
snapd:
  Installed: 2.59.1+23.04ubuntu1.2
  Candidate: 2.59.1+23.04ubuntu1.2
  Version table:
 *** 2.59.1+23.04ubuntu1.2 100
        100 http://192.168.10.7/ubuntu lunar-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2.59.1+23.04ubuntu1.1 500
        500 http://192.168.10.7/ubuntu lunar-updates/main amd64 Packages
        500 http://192.168.10.7/ubuntu lunar-security/main amd64 Packages
     2.59.1+23.04ubuntu1 500
        500 http://192.168.10.7/ubuntu lunar/main amd64 Packages
bdmurray@clean-lunar-amd64:~$ snap interfaces | grep fwupd
:fwupd snap-store

'snap interfaces' is deprecated; use 'snap connections'.

[Valentin David (valentin.david)]

I have a few questions while I am trying to reproduce the issue.

 * Are there any connections shown if running "snap connections --all"? (do not grep)
 * What happens after you restart snapd "sudo systemctl restart snapd"? Does the fwupd interface show up?
 * What is the output of "sudo journalctl -b0 -u snapd"?

[Valentin David (valentin.david)]

Also the content of `/etc/systemd/system/snap.snapd.*.mount` since I see it was not properly mounted.

[Valentin David (valentin.david)]

I have managed to reproduce the issue.

During upgrade, snapd deb gets "upgraded" to 2.60.4+23.10. Then a daemon-reload execute the broken snapd-generator. Then snapd gets restarted. snap.mount gets pulled in. The mounts get hidden. snapd does not find the files for /snap/snapd/* anymore and the install of firmware-updater fails.

The fix is to update the snapd deb in mantic.

As a work-around you can do "sudo mkdir /etc/systemd/system-generators; sudo ln -s /dev/null /etc/systemd/system-generators/snapd-generator" before upgrading.

[Brian Murray (brian-murray)]

Nick Rosbroook was able to complete an upgrade from Mantic to Noble and have firmware-updater installed after the upgrade. The process required a couple of hacks but those are well understood and would not have affected anything else. The hacks were:

1) modifying the dist-upgrade controller code to *not* disable -proposed
2) setting a pin priority for -proposed so that the snapd package from mantic-proposed was intalled

The full upgrade process can be found here:

https://dpaste.com/HZ4LQYECJ

[Nick Rosbrook (enr0n)]

Since snapd has been released to {lunar,mantic}-updates, I tested an upgrade from lunar to mantic without any modifications, and all went well:

nr@clean-lunar-amd64:~$ sudo apt update && sudo apt install snapd -y
[sudo] password for nr:
Get:1 http://security.ubuntu.com/ubuntu lunar-security InRelease [109 kB]
Hit:2 http://archive.ubuntu.com/ubuntu lunar InRelease
Get:3 http://archive.ubuntu.com/ubuntu lunar-updates InRelease [109 kB]
Get:4 http://security.ubuntu.com/ubuntu lunar-security/main amd64 DEP-11 Metadata [7,412 B]
Get:5 http://security.ubuntu.com/ubuntu lunar-security/universe amd64 DEP-11 Metadata [32.0 kB]
Get:6 http://archive.ubuntu.com/ubuntu lunar-updates/main Sources [155 kB]
Get:7 http://archive.ubuntu.com/ubuntu lunar-updates/main amd64 Packages [503 kB]
Get:8 http://archive.ubuntu.com/ubuntu lunar-updates/main amd64 DEP-11 Metadata [81.1 kB]
Get:9 http://archive.ubuntu.com/ubuntu lunar-updates/universe i386 Packages [674 kB]
Get:10 http://archive.ubuntu.com/ubuntu lunar-updates/universe amd64 Packages [814 kB]
Get:11 http://archive.ubuntu.com/ubuntu lunar-updates/universe amd64 DEP-11 Metadata [71.5 kB]
Fetched 2,555 kB in 2s (1,477 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-6.2.0-26 linux-headers-6.2.0-26-generic
  linux-image-5.19.0-46-generic linux-image-6.2.0-26-generic
  linux-modules-5.19.0-46-generic linux-modules-6.2.0-26-generic
  linux-modules-extra-5.19.0-46-generic linux-modules-extra-6.2.0-26-generic
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  snapd
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 26.2 MB of archives.
After this operation, 4,096 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu lunar-updates/main amd64 snapd amd64 2.59.1+23.04ubuntu1.2 [26.2 MB]
Fetched 26.2 MB in 2s (14.5 MB/s)
(Reading database ... 248469 files and directories currently installed.)
Preparing to unpack .../snapd_2.59.1+23.04ubuntu1.2_amd64.deb ...
Unpacking snapd (2.59.1+23.04ubuntu1.2) over (2.59.1+23.04ubuntu1.1) ...
Setting up snapd (2.59.1+23.04ubuntu1.2) ...
snapd.failure.service is a disabled or a static unit not running, not starting i
t.
snapd.snap-repair.service is a disabled or a static unit not running, not starti
ng it.
Failed to restart snapd.mounts-pre.target: Operation refused, unit snapd.mounts-
pre.target may be requested by dependency only (it is configured to refuse manua
l start/stop).
See system logs and 'systemctl status snapd.mounts-pre.target' for details.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 145.
Processing triggers for gnome-menus (3.36.0-1.1ubuntu1) ...
Processing triggers for man-db (2.11.2-1) ...
Processing triggers for dbus (1.14.4-1ubuntu1) ...
Processing triggers for mailcap (3.70+nmu1ubuntu1) ...
Processing triggers for desktop-file-utils (0.26-1ubuntu5) ....