Creating luks volumes from images using rbd fails if roots umask is 027

[Aymen Frikha]

When the cinder rbd driver converts the image to luks format it calls image_utils.convert_image with the default for run_as_root which is true. The converted image is owned by the root user with a group ownership of root. The rbd driver then overwrites the orignal file with the new converted file which has the affect of changing its ownership from cinder:cinder to root:root. The rbd driver then attempts to call rbd import as the cinder user but this fails as cinder cannot read the source file.

The following error appears in the cinder-volume log:

cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpry2juoj_

Full error log here: https://pastebin.ubuntu.com/p/BMS4vvBJy5/

Reproduce:

openstack volume type create \
    --encryption-provider nova.volume.encryptors.luks.LuksEncryptor \
    --encryption-cipher aes-xts-plain64 \
    --encryption-key-size 256 \
    --encryption-control-location front-end \
    LuksEncryptor-Template-256
openstack volume create \
    --type LuksEncryptor-Template-256 \
    test-vol-cirros-1 \
    --size 5 \
    --image cirros

This should work.

On cinder machine:
sed -i 's/^UMASK.*/UMASK 027/' /etc/login.defs

openstack volume create \
    --type LuksEncryptor-Template-256 \
    test-vol-cirros-2 \
    --size 5 \
    --image cirros

this will fail.

[Liam Young (gnuoy)]

The issue is the umask of the root user. With the CIS hardening that has been applied the file shown in the error message (/var/lib/cinder/conversion/tmpy88g_2co) is created with permissions:

-rw-r----- 1 root root 119508992 Jan 30 19:36 tmpk16n731p

On the cinder unit the following will allow volumes to be created:

cp /etc/login.defs /etc/login.defs.cis
sed -i 's/^UMASK.*/UMASK 022/' /etc/login.defs

[Liam Young (gnuoy)]

Just to be clear I am not recommending changing roots umask on a hardened system. Also, I think /etc/profile.d/Canonical_Ubuntu_CIS_rule-5.4.4.sh may need to be changed as well/instead of /etc/login.defs

[Aymen Frikha (aym-frikha)]

umask is also applied in bashrc and in /etc/profile.d

[Liam Young (gnuoy)]

To summarise:
1) /var/lib/cinder/conversion/ is created
drwxr-xr-x 2 cinder cinder 4096 Jan 30 19:22 /var/lib/cinder/conversion/

2) When root has a umask of 027 a temporary file is created in the above directory:
-rw-r----- 1 root root 119508992 Jan 31 10:43 tmpfzmu4__r

3) An "rbd import" is then run to import the above temporary file. This import command is run as cinder and fails as it is unable to read the temporary file.

This can be fixed on a hardened system without changing roots umask by setting the setgid bit on /var/lib/cinder/conversion. Also worth noting that the directory used for conversions is configurable via the cinder configuration option image_conversion_dir.

1) This could be set in the charm. The charm knows what image_conversion_dir is set to and could set the setgid bit on the directory.

2) The permissions could be set via a postinst package script. However the issue would reoccur if the user where to set image_conversion_dir to a non-default location.

3) Patch cinder to set the permissions of the temporary file. I don't know how practical this is since I think it would involve exposing chgrp via rootwrap.

*1 https://docs.openstack.org/cinder/zed/configuration/block-storage/samples/cinder.conf.html

[Liam Young (gnuoy)]

$ openstack volume create --type LuksEncryptor-Template-256 test-vol-cirros --size 5 --image cirros
...
$ openstack volume list
+--------------------------------------+-----------------+--------+------+-------------+
| ID | Name | Status | Size | Attached to |
+--------------------------------------+-----------------+--------+------+-------------+
| 73533d86-7d1d-4eee-ac15-c7d00749559d | test-vol-cirros | error | 5 | |
+--------------------------------------+-----------------+--------+------+-------------+
$ openstack volume delete test-vol-cirros
$ juju run --application cinder "chmod g+s /var/lib/cinder/conversion"

$ openstack volume create --type LuksEncryptor-Template-256 test-vol-cirros --size 5 --image cirros
...
$ openstack volume list
+--------------------------------------+-----------------+-----------+------+-------------+
| ID | Name | Status | Size | Attached to |
+--------------------------------------+-----------------+-----------+------+-------------+
| fa5b5f20-5381-462c-9e27-38378dc117a1 | test-vol-cirros | available | 5 | |
+--------------------------------------+-----------------+-----------+------+-------------+

[Liam Young (gnuoy)]

Having looked at the cinder code I think this warrants a patch to cinder itself.