i/b/dm-crypt: seccomp faults

[Peter Sabaini]

Using the dm-crypt connection I'm getting seccomp faults

[Wed Sep 6 13:10:46 2023] audit: type=1326 audit(1694005847.315:85): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=3278 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=141 compat=0 ip=0x7f2d3d86893b code=0x50000
[Wed Sep 6 13:10:53 2023] audit: type=1326 audit(1694005854.487:86): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=141 compat=0 ip=0x7f012932093b code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.179:87): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=249 compat=0 ip=0x7f0129324a3d code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.179:88): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=248 compat=0 ip=0x7f0129324a3d code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.183:89): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=249 compat=0 ip=0x7f0129324a3d code=0x50000

It seems like the assertions are not applied to existing processes immediately after connecting the dm-crypt interface. Restarting the microceph.daemon after interface connection works around this issue.

ubuntu@mc-3:~$ sudo microceph cluster bootstrap
ubuntu@mc-3:~$ sudo snap connect microceph:dm-crypt
ubuntu@mc-3:~$ sudo microceph disk add --encrypt /dev/vdc
Error: Failed adding new disk: Failed to open: Failed to luksOpen: /dev/disk/by-path/virtio-pci-0000:08:00.0, exit status 2, Failed to load key in kernel keyring.
No key available with this passphrase.

NOTE: OSD Encryption requires a snapd >= 2.59.1
Verify your version of snapd by running "snap version"

ubuntu@mc-3:~$ sudo dmesg -T | tail
...
[Wed Sep 6 13:10:46 2023] audit: type=1326 audit(1694005847.315:85): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=3278 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=141 compat=0 ip=0x7f2d3d86893b code=0x50000
[Wed Sep 6 13:10:53 2023] audit: type=1326 audit(1694005854.487:86): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=141 compat=0 ip=0x7f012932093b code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.179:87): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=249 compat=0 ip=0x7f0129324a3d code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.179:88): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=248 compat=0 ip=0x7f0129324a3d code=0x50000
[Wed Sep 6 13:10:55 2023] audit: type=1326 audit(1694005856.183:89): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=snap.microceph.daemon pid=4055 comm="cryptsetup" exe="/usr/sbin/cryptsetup" sig=0 arch=c000003e syscall=249 compat=0 ip=0x7f0129324a3d code=0x50000

ubuntu@mc-3:~$ sudo snap restart microceph.daemon
Restarted.
ubuntu@mc-3:~$ sudo microceph disk add --encrypt /dev/vdc
ubuntu@mc-3:~$ sudo microceph disk add --encrypt /dev/vdd
ubuntu@mc-3:~$ sudo microceph status
MicroCeph deployment summary:
- mc-3 (192.168.121.175)
  Services: mds, mgr, mon, osd
  Disks: 2

Is there an expectation that snap authors restart services after interface connection?

Versions:

snap 2.60.2
snapd 2.60.2
series 16
ubuntu 22.04
kernel 5.15.0-82-generic

For reference, debug output with SNAP_CONFINE_DEBUG=1
https://pastebin.canonical.com/p/Q9jdT2HZrs/

Steps to reproduce, using uvtools:

```
uvt-kvm create --cpu 4 --memory 8192 --disk 16 --host-passthrough --no-start "mc" release=jammy
virsh vol-create-as uvtool --format qcow2 "mc-virtio1.qcow" 34359738368
virsh attach-disk "mc" "/var/lib/uvtool/libvirt/images/mc-virtio1.qcow" vdc --subdriver qcow2 --targetbus virtio --config
virsh start mc
uvt-kvm wait "mc"
uvt-kvm ssh "mc" -- -t '\
  sudo snap install --channel latest/edge microceph
  sudo microceph cluster bootstrap
  sudo snap connect microceph:dm-crypt'
uvt-kvm ssh "mc" -- 'sudo microceph disk add --encrypt /dev/vdc'

```

[Sergio Cazzolato (sergio-j-cazzolato)]

Hi, thanks for raising this,

Reading the dm-crypt interface, I see the devices allowed to access through this one is:

/dev/dm-[0-9]* rwk,

I think you could try also connecting your snap to the block_devices interface which allows access to

# Access to raw devices, not individual partitions
/dev/hd[a-t] rwk, # IDE, MFM, RLL
/dev/sd{,[a-h]}[a-z] rwk, # SCSI
/dev/sdi[a-v] rwk, # SCSI continued
/dev/i2o/hd{,[a-c]}[a-z] rwk, # I2O hard disk
/dev/i2o/hdd[a-x] rwk, # I2O hard disk continued
/dev/mmcblk[0-9]{,[0-9],[0-9][0-9]} rwk, # MMC (up to 1000 devices)
/dev/vd[a-z] rwk, # virtio
/dev/loop[0-9]{,[0-9],[0-9][0-9]} rwk, # loopback (up to 1000 devices)
/dev/loop-control rw, # loopback control

[Ondrej Kubik (ondrak)]

As @Sergio correctly pointed out. This is misuse of dm-crypt interface.
dm-crypt interface is designed to access encrypted mapper devices as raw block devices.

As Sergio suggested, use block-device or consider the use of raw-volume, if you need access on the partition level.
cheers
Ondra

[Peter Sabaini (peter-sabaini)]

The microceph.daemon app already has block-devices permissions.

https://github.com/canonical/microceph/blob/895c58a56a600637261c17e6e12ac99731de403b/snap/snapcraft.yaml#L42

The permissions work fine too, it's just that they seem to require a restart of microceph.daemon after connecting dm-crypt. Is that expected? If so this maybe should be documented somewhere