Implementing secure nfs for cinder we have run into an issue as well. Using the deployment guide, root has been squashed and setuid disabled on the exports.
The problem results when trying to generate a cqow2 image from a boot cinder volume. This is run by the service cinder-volume.
What we see in the log is:
1. We generate a cqow2 image from a boot cinder volume: "volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72"
extracted from (/var/log/cinder/volume.log):
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.rpc.dispatcher ProcessExecutionError: Unexpected error while running command.
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.rpc.dispatcher Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.rpc.dispatcher Exit code: 1
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.rpc.dispatcher Stdout: u''
2016-10-28 11:01:40.419 1
2178 TRACE oslo_messaging.rpc.dispatcher Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72': Could not open '/var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72': Permission denied\n"
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.rpc.dispatcher
2. We observe the following permissions:
[root@cvs1lco03 mnt]# find /var/lib/cinder/mnt -name "*volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72*"
/var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b- 07dc46d0ec72
[root@cvs1lco03 mnt]# ls -ld /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72
-rw-rw----. 1 cinder cinder 21474836480 Oct 28 07:48 /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72
[root@cvs1lco03 mnt]# ls -ld /var/lib/cinder/mnt/
drwxr-xr-x. 5 cinder cinder 4096 Aug 19 12:39 /var/lib/cinder/mnt/
[root@cvs1lc
o03 mnt]# ls -ld /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc
drwxr-xr-x. 2 cinder cinder 4096 Oct 28 07:48 /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc
3. The problem is that the process that converts to the qcow2 image, uses the tool qemu-img which runs as root:
root 20776 20773 4 11:32 ? 00:00:00 /usr/bin/python2 /usr/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O qcow2 /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72 /var/lib/cinder/conversion/tmpqMzQ4L
root 20777 20776 24 11:32 ? 00:00:00 /bin/qemu-img convert -O qcow2 /var/lib/cinder/mnt/c257fa540e38bdfafef6dc8bb505b8dc/volume-5d0f7b70-f573-43dd-a78b-07dc46d0ec72 /var/lib/cinder/conversion/tmpqMzQ4L
We find that cinder uses “/usr/bin/cinder-rootwrap” to launch the process as root.
cinder ALL = (root) NOPASSWD: /us
r/bin/cinder-rootwrap /etc/cinder/rootwrap.conf *
To make the process run as cinder we have done the following:
We introduce a new optional parameter in a method in cinder python code:
"/usr/lib/python2.7/site-packages/cinder/volume/drivers/remotefs.py"
...
def copy_volume_to_image(self, context, volume, image_service, image_meta):
"""Copy the volume to the specified image.""" image_utils.upload_volume(context, image_service, image_meta,
self.local_path(volume), run_as_root=False)
.........
We added the last line "run_as_root=False".
Including this optional parameter, we can upload cinder-volumes to glance-images as "cinder":
Implementing secure nfs for cinder we have run into an issue as well. Using the deployment guide, root has been squashed and setuid disabled on the exports.
The problem results when trying to generate a cqow2 image from a boot cinder volume. This is run by the service cinder-volume.
What we see in the log is:
1. We generate a cqow2 image from a boot cinder volume: "volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72"
extracted from (/var/log/ cinder/ volume. log):
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging. rpc.dispatcher ProcessExecutio nError: Unexpected error while running command. rpc.dispatcher Command: sudo cinder-rootwrap /etc/cinder/ rootwrap. conf env LC_ALL=C qemu-img info /var/lib/ cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72 rpc.dispatcher Exit code: 1 rpc.dispatcher Stdout: u'' rpc.dispatcher Stderr: u"qemu-img: Could not open '/var/lib/ cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72': Could not open '/var/lib/ cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72': Permission denied\n" rpc.dispatcher
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.
2016-10-28 11:01:40.419 1
2178 TRACE oslo_messaging.
2016-10-28 11:01:40.419 12178 TRACE oslo_messaging.
2. We observe the following permissions: 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72*" cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b- 07dc46d0ec72 cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72 cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72 cinder/ mnt/ cinder/ mnt/ cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc
[root@cvs1lco03 mnt]# find /var/lib/cinder/mnt -name "*volume-
/var/lib/
[root@cvs1lco03 mnt]# ls -ld /var/lib/
-rw-rw----. 1 cinder cinder 21474836480 Oct 28 07:48 /var/lib/
[root@cvs1lco03 mnt]# ls -ld /var/lib/
drwxr-xr-x. 5 cinder cinder 4096 Aug 19 12:39 /var/lib/
[root@cvs1lc
o03 mnt]# ls -ld /var/lib/
drwxr-xr-x. 2 cinder cinder 4096 Oct 28 07:48 /var/lib/
3. The problem is that the process that converts to the qcow2 image, uses the tool qemu-img which runs as root: cinder- rootwrap /etc/cinder/ rootwrap. conf qemu-img convert -O qcow2 /var/lib/ cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72 /var/lib/ cinder/ conversion/ tmpqMzQ4L cinder/ mnt/c257fa540e3 8bdfafef6dc8bb5 05b8dc/ volume- 5d0f7b70- f573-43dd- a78b-07dc46d0ec 72 /var/lib/ cinder/ conversion/ tmpqMzQ4L
root 20776 20773 4 11:32 ? 00:00:00 /usr/bin/python2 /usr/bin/
root 20777 20776 24 11:32 ? 00:00:00 /bin/qemu-img convert -O qcow2 /var/lib/
We find that cinder uses “/usr/bin/ cinder- rootwrap” to launch the process as root.
( https:/ /wiki.openstack .org/wiki/ Rootwrap ) more info.
[root@cvs1lco03 mnt]# cat /etc/sudoers. d/cinder
Defaults:cinder !requiretty
cinder ALL = (root) NOPASSWD: /us rootwrap /etc/cinder/ rootwrap. conf *
r/bin/cinder-
To make the process run as cinder we have done the following:
We introduce a new optional parameter in a method in cinder python code: lib/python2. 7/site- packages/ cinder/ volume/ drivers/ remotefs. py" to_image( self, context, volume, image_service, image_meta):
image_ utils.upload_ volume( context,
image_service,
image_ meta, path(volume) ,
run_ as_root= False)
"/usr/
...
def copy_volume_
"""Copy the volume to the specified image."""
self.local_
.........
We added the last line "run_as_ root=False" .
Including this optional parameter, we can upload cinder-volumes to glance-images as "cinder":
ps -fe cinder/ mnt/785dce26ac9 aad019162c8de53 5ba4e6/ volume- 88f53fb3- 3514-4e82- bf5b-0e53ebcd87 f8 /var/lib/ cinder/ conversion/ tmphsyBEL
cinder 27448 20682 0 14:08 ? 00:00:00 qemu-img convert -O qcow2 /var/lib/
Can this be added to the driver?