Comment 4 for bug 1708629

Revision history for this message
Pranali Deore (pranali-deore) wrote :

Seems like this issue is not yet been fixed :(.
Something might have been changed after this merge or may be I have tested it in wrong way while resolving the firewall issues :(

Anyway,

Now the issue here is, there are some permission issues with the mounted volume dir /var/lib/glance and even after resolving this permission issue, the original issue doesn't get resolved i.e.,
The image would not be created in the share as the NFS endpoint is not getting mounted successfully on the container. .
I have investigated little more on this and noticed that the mounting of NFS endpoint[1] doesn't work inside the container even though it makes mountpoint entry in /etc/fstab.

"----------------------------------------------------------------------------------------------------------------------------
()[glance@overcloud-controller-0 /]$ cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Tue Aug 1 17:23:07 2017
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=1af758b4-34e7-40f6-a12d-9ee83d3d521e / ext4 defaults 1 1
192.168.24.1:/export/glance /var/lib/glance/images nfs4 _netdev,bg,intr,context=system_u:object_r:glance_var_lib_t:s0 0 0
----------------------------------------------------------------------------------------------------------------------------"

I wonder, why the OC deployment doesn't fail when it fails to mount NFS endpoint on container or it doens't even try to mount? (I couldn't find anything related to it in logs :()
I'hv fould only this much logs related to nfs, mounting endpoint is missing:

Sep 7 15:23:56 localhost puppet-user[10]: (/Stage[main]/Tripleo::Glance::Nfs_mount/File[/var/lib/glance/images]/ensure) created
Sep 7 15:23:56 localhost puppet-user[10]: (/Stage[main]/Tripleo::Glance::Nfs_mount/File_line[NFS for glance in fstab]/ensure) created
Sep 7 15:23:56 localhost journal: Notice: /Stage[main]/Tripleo::Glance::Nfs_mount/File[/var/lib/glance/images]/ensure: created
Sep 7 15:23:56 localhost journal: Notice: /Stage[main]/Tripleo::Glance::Nfs_mount/File_line[NFS for glance in fstab]/ensure: created

I'm new in tripleo and don't have hands-on knowledge and experience on containers but after doing some research on this,
I suspect this mounting NFS endpoint on container might not be happenning due to below reasons:
1. nfs-utils is not installed in the container
2. container is not previledged and it runs with non-root user(glance)
3. additionally I think, automount for NFS volume (entry in fstab) doesn't work on container restart

IMO, any of the following could be used for fixing this issue (Please correct me if I'm wrong)
1. mount NFS endpoint on the host ( adding mount in host-prep-tasks section) and fix the permission issue.
2. Create glance_api container with nfs-utils and run it in previledged mode with root user(in this case, mounted NFS volume will not be needed) and we would need to modify kolla start to mount entries in fstab
3. fall back to non-containerized similar to cinder[2].
4. or could be possible by using the docker volume NFS plugin but I'ven't tried this yet.

[1]: https://github.com/openstack/puppet-tripleo/blob/07500da43e6a991511099a37106015037da159b8/manifests/glance/nfs_mount.pp#L66-L70
[2]: https://bugs.launchpad.net/tripleo/+bug/1698136