Unable to use multiprocessing, this is normally caused by lack of access to /dev/shm
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Triaged
|
Medium
|
Unassigned |
Bug Description
Summary:
==================
Deployment of stein overcloud fails with Stderr: u'ERROR! Unable to use multiprocessing, this is normally caused by lack of access to /dev/shm: [Errno 13] Permission denied\n'
Steps to reproduce
==================
Deploy openstack stein containerized deployment
Expected result
===============
Overcloud deployed successfully.
Actual result
=============
Full deployment command: /usr/bin/python2 /usr/bin/openstack cbis overcloud deploy --templates templates/ --no-cleanup
Ansible Command: ansible-playbook-2 -vvvvv /tmp/ansible-
fails during the deployment with exit code 1.
================
Quick troubleshooting shows correct permissions of shm.
(undercloud) [stack@undercloud cbis]$ ls -lha /dev/ | grep shm
drwxrwxrwt. 2 root root 60 Dec 14 22:04 shm
(undercloud) [stack@undercloud cbis]$ ls -lha /dev/shm
total 84K
drwxrwxrwt. 2 root root 60 Dec 14 22:04 .
drwxr-xr-x. 19 root root 3.2K Dec 14 22:27 ..
-rw-------. 1 root root 81K Dec 14 20:35 libpod_lock
Environment
===========
Stein with Centos 7.7
3 Controllers , 2 ovs computes.
ceph version 12.2.11 (26dc3775efc7bb
ansible 2.8.4
config file = /etc/ansible/
configured module search path = [u'/home/
ansible python module location = /usr/lib/
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Aug 7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Which networking type did you use?
Openvswitch
Logs & Configs
==============
Stdout: u"ansible-
Stderr: u'ERROR! Unable to use multiprocessing, this is normally caused by lack of access to /dev/shm: [Errno 13] Permission denied\n'
description: | updated |
Changed in tripleo: | |
assignee: | Emilien Macchi (emilienm) → nobody |
Changed in tripleo: | |
milestone: | ussuri-3 → ussuri-rc3 |
Changed in tripleo: | |
milestone: | ussuri-rc3 → victoria-1 |
Changed in tripleo: | |
milestone: | victoria-1 → victoria-3 |
I'm facing exact same problem, which seems to be intermittent. I have noticed, that /dev/shm directory inside mistral containers is not writable (even by root on the container). I believe that's because it is mounted incorrectly to system partition (and blocked by SELinux).
On undercloud with the bug:
(inside mistral_executor) df -h
...
dev/vda10 219G 42G 177G 20% /dev/shm
...
On undercloud without the bug:
(inside mistral_executor) df -h
...
shm 63M 0 63M 0% /dev/shm
...