virsh driver creates unusable disks in LVM pools

[Hector Martin]

maas calls virsh vol-create-as with --allocation=0, which is not usable on LVM pools. It will create thinly provisioned snapshot volume in the pool of size 4MiB. By default, LVM thinly provisioned volumes are not auto-extended, so as soon as 4MiB of data are written to the volume it becomes invalid and breaks. Even if auto-extending is configured in lvm.conf and dmeventd is running, it cannot extend the volume quickly enough before it becomes full and invalidated.

I'd expect LVM-backed volumes to not be silently thinly provisioned, since it doesn't work like qcow2 and its ilk - it relies on dmeventd to be running and configured, and if it isn't (or if it stops working, or if it doesn't react in time) then your volumes just break completely when the snapshots get full. But if this behavior is desired, it should allocate at least something reasonable so that dmeventd has a chance to keep the snapshot allocation ahead of actual usage. In my testing, 1G was just barely enough to keep up with a normal install process (256MB was insufficient). Personally I don't think it's worth the risk, this is very fragile and it seems dmeventd is quite slow at reacting and resizing the volume (takes a few seconds sometimes).

To reproduce just use the virsh backend with an LVM-backed libvirt storage pool, and compose a host. Composing itself will work, but if you actually try to install anything or touch the disk it will quickly break and throw I/O errors. lvs shows the volume as being provisioned with 4MiB of storage (one extent), and even if auto-extension is enabled it just breaks like this:

Oct 29 22:48:48 admin01 lvm[32701]: Size of logical volume vg-admin/c9eaf348-941f-4959-8a5b-819eaabd567b changed from 4.00 MiB (1 extents) to 8.00 MiB (2 extents).
Oct 29 22:48:48 admin01 lvm[32701]: Logical volume vg-admin/c9eaf348-941f-4959-8a5b-819eaabd567b successfully resized.
Oct 29 22:48:48 admin01 kernel: [5364243.357505] device-mapper: snapshots: Invalidating snapshot: Unable to allocate exception.
Oct 29 22:48:48 admin01 lvm[32701]: WARNING: Snapshot vg--admin-c9eaf348--941f--4959--8a5b--819eaabd567b changed state to: Invalid and should be removed.

A manual workaround is to issue `lvresize -L 1G <vg> <name>` right after maas creates the machine.

$ dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-==============================-============-=================================================
ii maas 2.4.2-7034-g2f5deb8b8-0ubuntu1 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server common files
ii maas-dhcp 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS DHCP server
un maas-dns <none> <none> (no description available)
ii maas-proxy 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS Caching Proxy
ii maas-rack-controller 2.4.2-7034-g2f5deb8b8-0ubuntu1 all Rack Controller for MAAS
ii maas-region-api 2.4.2-7034-g2f5deb8b8-0ubuntu1 all Region controller API service for MAAS
ii maas-region-controller 2.4.2-7034-g2f5deb8b8-0ubuntu1 all Region Controller for MAAS
un maas-region-controller-min <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisioningserver <none> <none> (no description available)
ii python3-django-maas 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server provisioning libraries (Python 3)