LXD 5.0.3 does not handle root disk sizes in ZFS correctly

[Bartosz Woronicz]

Hi, I encounter issue when I manually override the disk size. I got real ZFS backed pool

The snap version is 5.0.3 from 5.0/stable channel
$ snap info lxd |grep install
 installed: 5.0.3-babaaf8 (27948) 91MB -

When specify 24GB it's ok

mastier@drakkar:~$ lxc init ubuntu:22.04 testvm --vm -c limits.cpu=1 -c limits.memory=2GiB
mastier@drakkar:~$ lxc config device override testvm root size=24GB
Device root overridden for testvm
mastier@drakkar:~$ lxc config device list testvm
root
mastier@drakkar:~$ lxc start testvm
mastier@drakkar:~$ lxc exec testvm bash
root@testvm:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 22G 1.4G 21G 7% /
tmpfs 991M 0 991M 0% /dev/shm
tmpfs 397M 612K 396M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 50M 14M 37M 27% /run/lxd_agent
/dev/sda15 105M 6.1M 99M 6% /boot/efi
root@testvm:~#
exit
mastier@drakkar:~$ lxc delete --force testvm

When I set 15 or 16GB.

mastier@drakkar:~$ lxc init ubuntu:22.04 testvm --vm -c limits.cpu=1 -c limits.memory=2GiB
Creating testvm

mastier@drakkar:~$ lxc config device override esa-vpn root size=15GB
Error: Failed to update device "root": Failed to run: zfs set volsize=15000002560 rpool/lxd/virtual-machines/esa-vpn.block: exit status 255 (cannot set property for 'rpool/lxd/virtual-machines/esa-vpn.block': 'volsize' must be a multiple of volume block size (16K))

mastier@drakkar:~$ lxc config device override testvm root size=16GB
Error: Failed to update device "root": Failed to run: zfs set volsize=16000000000 rpool/lxd/virtual-machines/testvm.block: exit status 255 (cannot set property for 'rpool/lxd/virtual-machines/testvm.block' 'volsize' must be a multiple of volume block size (16K))

[Simon Déziel (sdeziel)]

@Bartosz it seems you were lucky with 24GB ends being with a multiple of 16KiB (ZFS uses K and G to mean KiB and GiB btw):

$ lxc config device override testvm root size=24GB
Device root overridden for testvm
$ zfs get -p volsize default/virtual-machines/testvm.block
NAME PROPERTY VALUE SOURCE
default/virtual-machines/testvm.block volsize 24000004096 local

15 and 16GB are not multiple of 16KiB. You can generally avoid such issues by defining the storage sizes by using MiB or GiB units. Like here:

$ lxc config device override testvm root size=15GiB
Device root overridden for testvm
$ lxc config device set testvm root size=16GiB

Could you check if using GiB works for you too? I'll mark the bug as incomplete until you can test and report. Thank you.

[Bartosz Woronicz (mastier1)]

not really lucky, 11 or 12GB also works

mastier@drakkar:~$ lxc config device override esa-vpn root size=15GiB
Device root overridden for esa-vpn

that's kinda solution here :-)
Yet maybe we could add this hint here in the error message.