lxd-installer: permission error not handled and lxd snap not installed

[Nick Rosbrook]

[ Impact ]

* If the LXD group is not pre-created and/or the primary user isn't member of it, lxd-installer will fail to install LXD with a cryptic permission denied error and a long delay before complaining the command is not found.

[ Test Plan ]

Create a test instance:

$ lxc launch ubuntu-minimal-daily:24.04 u1

Make sure LXD snap is not installed:

$ ! lxc exec u1 -- snap list lxd || false
error: no matching snaps installed

Remove the LXD group:

$ lxc exec u1 -- delgroup lxd
warn: The group `lxd' does not exist.

Check lxd-installer version (update to -proposed package if needed):

$ lxc exec u1 -- dpkg -l | grep lxd-installer
ii lxd-installer 4 all Wrapper to install lxd snap on demand

Trigger lxd-installer as the regular user not a member of the lxd group:

$ lxc exec --user 1000 --group 1000 --env HOME=/home/ubuntu u1 -- lxc list
Installing LXD snap, please be patient.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
PermissionError: [Errno 13] Permission denied
/usr/sbin/lxc: 12: exec: /snap/bin/lxc: not found
Error: Command not found

The output above is the cryptic error that should not be displayed if the updated package is installed. Instead, a user not in the lxd group that would trigger the lxd-installer, should immediately receive this nicer error message:

$ lxc exec --user 1000 --group 1000 --env HOME=/home/ubuntu u1 -- lxc list
Unable to trigger the installation of the LXD snap.
Please make sure you're a member of the 'lxd' system group.

And the return code should be 1 to indicate an error.

[ Where problems could occur ]

The proposed fix is to check if the /run/lxd-installer.socket socket is writable and if not, report a nicer error and exist with an error before trying to do the installation and eventually error out.

It is possible for this socket to either be missing or not be writable for a reason different than the invoking user not being in the lxd group.

If that's the case, the hint to verify the group membership would be misleading.

[Original description]

On a fresh install of 24.04 server on Raspberry Pi, I ran 'lxc list' without having the lxd snap installed already. There was an attempt to install the snap, but it failed, and the snap needed to be manually installed:

nr@pi5:~$ lxc list
Installing LXD snap, please be patient.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
PermissionError: [Errno 13] Permission denied
/usr/sbin/lxc: 12: exec: /snap/bin/lxc: not found

More details:

nr@pi5:~$ cat /etc/os-release
PRETTY_NAME="Ubuntu Noble Numbat (development branch)"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
nr@pi5:~$ apt policy lxd-installer
lxd-installer:
  Installed: 4
  Candidate: 4
  Version table:
 *** 4 500
        500 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 Packages
        100 /var/lib/dpkg/status

[Simon Déziel (sdeziel)]

@enr0n, could you share `snap changes` as well as `snap info lxd`, please?

[Nick Rosbrook (enr0n)]

nr@pi5:~$ snap info lxd
name: lxd
summary: LXD - container and VM manager
publisher: Canonical✓
store-url: https://snapcraft.io/lxd
contact: https://github.com/canonical/lxd/issues
license: AGPL-3.0
description: |
  LXD is a system container and virtual machine manager.

  It offers a simple CLI and REST API to manage local or remote instances,
  uses an image based workflow and support for a variety of advanced features.

  Images are available for all Ubuntu releases and architectures as well
  as for a wide number of other Linux distributions. Existing
  integrations with many deployment and operation tools, makes it work
  just like a public cloud, except everything is under your control.

  LXD containers are lightweight, secure by default and a great
  alternative to virtual machines when running Linux on Linux.

  LXD virtual machines are modern and secure, using UEFI and secure-boot
  by default and a great choice when a different kernel or operating
  system is needed.

  With clustering, up to 50 LXD servers can be easily joined and managed
  together with the same tools and APIs and without needing any external
  dependencies.

  Supported configuration options for the snap (snap set lxd [<key>=<value>...]):

    - ceph.builtin: Use snap-specific Ceph configuration [default=false]
    - ceph.external: Use the system's ceph tools (ignores ceph.builtin) [default=false]
    - criu.enable: Enable experimental live-migration support [default=false]
    - daemon.debug: Increase logging to debug level [default=false]
    - daemon.group: Set group of users that have full control over LXD [default=lxd]
    - daemon.user.group: Set group of users that have restricted LXD access [default=lxd]
    - daemon.preseed: Pass a YAML configuration to `lxd init` on initial start
    - daemon.syslog: Send LXD log events to syslog [default=false]
    - daemon.verbose: Increase logging to verbose level [default=false]
    - lvm.external: Use the system's LVM tools [default=false]
    - lxcfs.pidfd: Start per-container process tracking [default=false]
    - lxcfs.loadavg: Start tracking per-container load average [default=false]
    - lxcfs.cfs: Consider CPU shares for CPU usage [default=false]
    - lxcfs.debug: Increase logging to debug level [default=false]
    - openvswitch.builtin: Run a snap-specific OVS daemon [default=false]
    - openvswitch.external: Use the system's OVS tools (ignores openvswitch.builtin) [default=false]
    - ovn.builtin: Use snap-specific OVN configuration [default=false]
    - ui.enable: Enable the web interface [default=false]

  For system-wide configuration of the CLI, place your configuration in
  /var/snap/lxd/common/global-conf/ (config.yml and servercerts)
commands:
  - lxd.buginfo
  - lxd.check-kernel
  - lxd.lxc
  - lxd
services:
  lxd.activate: oneshot, enabled, inactive
  lxd.daemon: simple, enabled, inactive
  lxd.user-daemon: simple, enabled, inactive
snap-id: J60k4JY0HppjwOjW8dZdYc8obXKxujRu
tracking: 5.21/stable
refresh-date: yesterday at 20:04 UTC
channels:
  5.21/stable: 5.21.1-3f3c2a9 2024-04-09 (28163) 97MB -
  5.21/candidate: 5.21.1-3f3c2a9 2024-04-09...

[Nick Rosbrook (enr0n)]

For the record, the permission error makes sense given the SocketUser=root SocketGroup=lxd in lxd-installer.socket. I just think it would be nice if the error was handled gracefully and explained to the user, rather than having the command hang for a while.

[Simon Déziel (sdeziel)]

The problem is not with a Raspberry Pi as that works fine:

ubuntu@ubuntu:~$ cat /etc/os-release
PRETTY_NAME="Ubuntu Noble Numbat (development branch)"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
ubuntu@ubuntu:~$ apt policy lxd-installer
lxd-installer:
  Installed: 4
  Candidate: 4
  Version table:
 *** 4 500
        500 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 Packages
        100 /var/lib/dpkg/status

ubuntu@ubuntu:~$ snap list
Name Version Rev Tracking Publisher Notes
snapd 2.61.2 21185 latest/stable canonical✓ snapd

ubuntu@ubuntu:~$ lxc ls
Installing LXD snap, please be patient.
If this is your first time running LXD on this machine, you should also run: lxd init
To start your first container, try: lxc launch ubuntu:22.04
Or for a virtual machine: lxc launch ubuntu:22.04 --vm

+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

ubuntu@ubuntu:~$ snap changes
ID Status Spawn Ready Summary
1 Done today at 03:16 UTC today at 13:50 UTC Initialize system state
2 Done today at 13:50 UTC today at 13:50 UTC Initialize device
3 Done today at 14:00 UTC today at 14:02 UTC Install "lxd" snap from "5.21/stable/ubuntu-24.04" channel

ubuntu@ubuntu:~$ snap list lxd
Name Version Rev Tracking Publisher Notes
lxd 5.21.1-3f3c2a9 28163 5.21/stable/… canonical✓ -

The problem seems to be when the invoking user is not a member of the `lxd` group which is why there is a Permission denied.

[Steve Langasek (vorlon)]

This is in the noble queue, which is now for SRUs only. If your intention is for this to be included as an SRU, please follow the SRU template at https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxd-installer - 5

---------------
lxd-installer (5) oracular; urgency=medium

  * scripts/lxc: check if socket is writeable (LP: #2061017)
  * scripts/lxc: give time to snapd to make command available
  * d/tests/control: remove workaround for LP: #2046379
  * d/tests/not-member-of-lxd-group: new test
  * lxd-installer-service: fallback to pulling LXD from default channel
    (LP: #2061910)
  * d/tests/install-on-demand: update fallback case

 -- Simon Deziel <email address hidden> Tue, 16 Apr 2024 17:46:08 -0400

[Timo Aaltonen (tjaalton)]

Hello Nick, or anyone else affected,

Accepted lxd-installer into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxd-installer/4ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Simon Déziel (sdeziel)]

Noble SRU verification:

Download the package from launchpad (not yet published in -proposed):

$ wget https://launchpad.net/ubuntu/+source/lxd-installer/4ubuntu0.1/+build/28436117/+files/lxd-installer_4ubuntu0.1_all.deb

$ lxc launch ubuntu-minimal-daily:24.04 u1

$ lxc file push /tmp/lxd-installer_4ubuntu0.1_all.deb u1/tmp/

$ lxc exec u1 -- apt-get install /tmp/lxd-installer_4ubuntu0.1_all.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'lxd-installer' instead of '/tmp/lxd-installer_4ubuntu0.1_all.deb'
The following packages will be upgraded:
  lxd-installer
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/3926 B of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /tmp/lxd-installer_4ubuntu0.1_all.deb lxd-installer all 4ubuntu0.1 [3926 B]
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 12551 files and directories currently installed.)
Preparing to unpack .../lxd-installer_4ubuntu0.1_all.deb ...
Unpacking lxd-installer (4ubuntu0.1) over (4) ...
Setting up lxd-installer (4ubuntu0.1) ...

$ lxc exec u1 -- deluser ubuntu lxd
info: Removing user `ubuntu' from group `lxd' ...

$ lxc exec --user 1000 --group 1000 --env HOME=/home/ubuntu u1 -- lxc ls
Unable to trigger the installation of the LXD snap.
Please make sure you're a member of the 'lxd' system group.

This last step shows a clearer error to the user, marking as verified.