Load thin Kernel Modules in initramfs for support on boot

[Axton Grams]

I have an issue where lvm2 thinpool volume groups are not automatically activated after reboot.

Environmentals:
Ubuntu Server 15.10, minimal install
UEFI/Secure Boot in use
Ubuntu 15.10 (GNU/Linux 4.2.0-25-generic x86_64)

root@cluster-02:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="15.10 (Wily Werewolf)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 15.10"
VERSION_ID="15.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

Here is the volume config before adding the new volume:

root@cluster-02:~# lvs -a
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  lvswap vgraid0 -wi-ao---- 29.80g
  lvtmp vgraid0 -wi-ao---- 29.80g
  lvvartmp vgraid0 -wi-ao---- 29.80g
  lvhome vgraid10 -wi-ao---- 29.80g
  lvroot vgraid10 -wi-ao---- 7.45g
  lvusr vgraid10 -wi-ao---- 7.45g
  lvvar vgraid10 -wi-ao---- 3.72g
  lvvarcache vgraid10 -wi-ao---- 119.21g
  lvvarlib vgraid10 -wi-ao---- 32.00g
  lvvarlog vgraid10 -wi-ao---- 14.90g

I add a new thinpool volume using this command:

lvcreate -L 1T --type thin-pool --thinpool vgraid10/lvlxc

root@cluster-02:~# lvcreate -L 1T --type thin-pool --thinpool vgraid10/lvlxc
  Logical volume "lvlxc" created.

Which results in this lvs:

root@cluster-02:~# lvs -a
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  lvswap vgraid0 -wi-ao---- 29.80g
  lvtmp vgraid0 -wi-ao---- 29.80g
  lvvartmp vgraid0 -wi-ao---- 29.80g
  lvhome vgraid10 -wi-ao---- 29.80g
  lvlxc vgraid10 twi-a-tz-- 1.00t 0.00 0.42
  [lvlxc_tdata] vgraid10 Twi-ao---- 1.00t
  [lvlxc_tmeta] vgraid10 ewi-ao---- 128.00m
  [lvol0_pmspare] vgraid10 ewi------- 128.00m
  lvroot vgraid10 -wi-ao---- 7.45g
  lvusr vgraid10 -wi-ao---- 7.45g
  lvvar vgraid10 -wi-ao---- 3.72g
  lvvarcache vgraid10 -wi-ao---- 119.21g
  lvvarlib vgraid10 -wi-ao---- 32.00g
  lvvarlog vgraid10 -wi-ao---- 14.90g

I then create an unprivileged lxc container using the thinpool:

root@cluster-02:~# lxc-create -B lvm --vgname=vgraid10 --thinpool=lvlxc -t download -n tmpl-centos-7-unpriv --fssize 16GB -- -d centos -r 7 -a amd64
File descriptor 3 (/var/lib/lxc/tmpl-centos-7-unpriv/partial) leaked on lvcreate invocation. Parent PID 9118: lxc-create
  Logical volume "tmpl-centos-7-unpriv" created.
Using image from local cache
Unpacking the rootfs
...

The lvs output:
root@cluster-02:~# lvs -a
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  lvswap vgraid0 -wi-ao---- 29.80g
  lvtmp vgraid0 -wi-ao---- 29.80g
  lvvartmp vgraid0 -wi-ao---- 29.80g
  lvhome vgraid10 -wi-ao---- 29.80g
  lvlxc vgraid10 twi-aotz-- 1.00t 0.09 0.46
  [lvlxc_tdata] vgraid10 Twi-ao---- 1.00t
  [lvlxc_tmeta] vgraid10 ewi-ao---- 128.00m
  [lvol0_pmspare] vgraid10 ewi------- 128.00m
  lvroot vgraid10 -wi-ao---- 7.45g
  lvusr vgraid10 -wi-ao---- 7.45g
  lvvar vgraid10 -wi-ao---- 3.72g
  lvvarcache vgraid10 -wi-ao---- 119.21g
  lvvarlib vgraid10 -wi-ao---- 32.00g
  lvvarlog vgraid10 -wi-ao---- 14.90g
  tmpl-centos-7-unpriv vgraid10 Vwi-a-tz-- 16.00g lvlxc 5.94

Everything is ok at this point. Now, I will reboot the machine.

root@cluster-02:~# lvs -a
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  lvswap vgraid0 -wi-ao---- 29.80g
  lvtmp vgraid0 -wi-ao---- 29.80g
  lvvartmp vgraid0 -wi-ao---- 29.80g
  lvhome vgraid10 -wi-ao---- 29.80g
  lvlxc vgraid10 twi---tz-- 1.00t
  [lvlxc_tdata] vgraid10 Twi------- 1.00t
  [lvlxc_tmeta] vgraid10 ewi------- 128.00m
  [lvol0_pmspare] vgraid10 ewi------- 128.00m
  lvroot vgraid10 -wi-ao---- 7.45g
  lvusr vgraid10 -wi-ao---- 7.45g
  lvvar vgraid10 -wi-ao---- 3.72g
  lvvarcache vgraid10 -wi-ao---- 119.21g
  lvvarlib vgraid10 -wi-ao---- 32.00g
  lvvarlog vgraid10 -wi-ao---- 14.90g
  tmpl-centos-7-unpriv vgraid10 Vwi---tz-- 16.00g lvlxc

At this point, the volume groups (thinpool and thin volume) are not active. This causes issues and requires that I manually activate the volumes:

root@cluster-02:~# lvchange -ay vgraid10/lvlxc
root@cluster-02:~# lvs -a
  LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
  lvswap vgraid0 -wi-ao---- 29.80g
  lvtmp vgraid0 -wi-ao---- 29.80g
  lvvartmp vgraid0 -wi-ao---- 29.80g
  lvhome vgraid10 -wi-ao---- 29.80g
  lvlxc vgraid10 twi-aotz-- 1.00t 0.09 0.46
  [lvlxc_tdata] vgraid10 Twi-ao---- 1.00t
  [lvlxc_tmeta] vgraid10 ewi-ao---- 128.00m
  [lvol0_pmspare] vgraid10 ewi------- 128.00m
  lvroot vgraid10 -wi-ao---- 7.45g
  lvusr vgraid10 -wi-ao---- 7.45g
  lvvar vgraid10 -wi-ao---- 3.72g
  lvvarcache vgraid10 -wi-ao---- 119.21g
  lvvarlib vgraid10 -wi-ao---- 32.00g
  lvvarlog vgraid10 -wi-ao---- 14.90g
  tmpl-centos-7-unpriv vgraid10 Vwi---tz-- 16.00g lvlxc

I have tried setting flags on the thinpool (vgraid10/lvlxc) as follows, to no avail:

root@cluster-02:~# lvchange -kn vgraid10/lvlxc
root@cluster-02:~# lvchange -ay vgraid10/lvlxc
root@cluster-02:~# lvchange -aye vgraid10/lvlxc

[Axton Grams (axton-grams)]

root@cluster-02:~# apt-cache policy thin-provisioning-tools
thin-provisioning-tools:
  Installed: 0.3.2-1ubuntu1
  Candidate: 0.3.2-1ubuntu1
  Version table:
 *** 0.3.2-1ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ wily/universe amd64 Packages
        100 /var/lib/dpkg/status

[Axton Grams (axton-grams)]

I found a temporary solution that seems ok for now. This article walks through the addition of an initramfs hook to add the necessary kernel modules and and executables to the initramfs image so that the boot process can deal with lvm thinpools and thinvolumes.

http://askubuntu.com/questions/673815/how-do-i-start-my-laptop-with-root-partition-on-lvm2-thin-pool

To recap, create file /etc/initramfs-tools/hooks/thin-provisioning-tools:

#!/bin/sh

PREREQ="lvm2"

prereqs()
{
        echo ""
}

case $1 in
prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /usr/sbin/thin_check
copy_exec /usr/sbin/thin_dump
copy_exec /usr/sbin/thin_repair
copy_exec /usr/sbin/thin_restore
copy_exec /sbin/dmeventd

manual_add_modules dm_thin_pool

Then update the initramfs image:
# chmod 755 /etc/initramfs-tools/hooks/thin-provisioning-tools
# update-initramfs-tools -u

After reboot, things work as expected. I expect the thin-provisioning-tools package should be updated to make these changes. What is the right way to report this as an issue with this package?

[Axton Grams (axton-grams)]

This addresses some issues with the previous initramfs changes where some necessary libs were missing. This works well and cleans up some errors with dmeventd when using the previous initramfs changes.

root@cluster-01:~# vi /etc/initramfs-tools/hooks/thin-provisioning-tools
#!/bin/sh

PREREQ="lvm2"

prereqs()
{
 echo ""
}

case $1 in
prereqs)
 prereqs
 exit 0
 ;;
esac

. /usr/share/initramfs-tools/hook-functions

copy_exec /usr/sbin/thin_check
copy_exec /usr/sbin/thin_dump
copy_exec /usr/sbin/thin_repair
copy_exec /usr/sbin/thin_restore
copy_exec /sbin/dmeventd
copy_exec /lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2thin.so
copy_exec /lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2.so.2.02
copy_exec /lib/x86_64-linux-gnu/liblvm2cmd.so.2.02

manual_add_modules dm_thin_pool

[Axton Grams (axton-grams)]

The files missing from the initramfs are from package dmeventd, not thin-provisioning-tools. As such, these changes to the initramfs should instead be included in the dmeventd package.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Travisgevans (travisgevans)]

I was able to use this hook to get Ubuntu 15.10 installed and running from a thin LVM volume in a VirtualBox VM.

However, the lvm2-monitor service seems wonky if started from the initramfs, strange errors like:

<volumegroup>-<lvpool>-tpool: event registration failed: 2561:11 libdevmapper-event-lvm2thin.so dlopen failed: /lib/x86_64-linux-gnu/device-mapper/libdevmapper-event-lvm2thin.so: cannot open shared object file: No such file or directory

even though the library is both in the root fs and the initrd. Leaving dmeventd and its libraries out of the initramfs seems to allow it to function normally.

[Steve Dodd (anarchetic)]

Looks like this may finally have been fixed in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774560