manila dsvm jobs fail to setup horizon dashboard due to Permission Denied error on lock file

[Matt Riedemann]

Seen here:

IOError: [Errno 13] Permission denied: '/opt/stack/new/devstack/src/horizon/openstack_dashboard/local/_opt_stack_new_devstack_src_horizon_openstack_dashboard_local_.secret_key_store.lock'

2016-01-08 18:46:43.868 | Traceback (most recent call last):
2016-01-08 18:46:43.868 | File "/usr/local/bin/django-admin", line 11, in <module>
2016-01-08 18:46:43.868 | sys.exit(execute_from_command_line())
2016-01-08 18:46:43.868 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
2016-01-08 18:46:43.868 | utility.execute()
2016-01-08 18:46:43.868 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 303, in execute
2016-01-08 18:46:43.868 | settings.INSTALLED_APPS
2016-01-08 18:46:43.868 | File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 48, in __getattr__
2016-01-08 18:46:43.868 | self._setup(name)
2016-01-08 18:46:43.869 | File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 44, in _setup
2016-01-08 18:46:43.869 | self._wrapped = Settings(settings_module)
2016-01-08 18:46:43.869 | File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 92, in __init__
2016-01-08 18:46:43.869 | mod = importlib.import_module(self.SETTINGS_MODULE)
2016-01-08 18:46:43.869 | File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2016-01-08 18:46:43.869 | __import__(name)
2016-01-08 18:46:43.869 | File "/opt/stack/new/devstack/src/horizon/openstack_dashboard/settings.py", line 342, in <module>
2016-01-08 18:46:43.870 | '.secret_key_store'))
2016-01-08 18:46:43.870 | File "/opt/stack/new/devstack/src/horizon/horizon/utils/secret_key.py", line 54, in generate_or_read_from_file
2016-01-08 18:46:43.870 | with lock:
2016-01-08 18:46:43.870 | File "/usr/local/lib/python2.7/dist-packages/fasteners/process_lock.py", line 174, in __enter__
2016-01-08 18:46:43.870 | self.acquire()
2016-01-08 18:46:43.870 | File "/usr/local/lib/python2.7/dist-packages/fasteners/process_lock.py", line 151, in acquire
2016-01-08 18:46:43.870 | self._do_open()
2016-01-08 18:46:43.870 | File "/usr/local/lib/python2.7/dist-packages/fasteners/process_lock.py", line 123, in _do_open
2016-01-08 18:46:43.870 | self.lockfile = open(self.path, 'a')
2016-01-08 18:46:43.870 | IOError: [Errno 13] Permission denied: '/opt/stack/new/devstack/src/horizon/openstack_dashboard/local/_opt_stack_new_devstack_src_horizon_openstack_dashboard_local_.secret_key_store.lock'

Looks like this might have been the change in devstack that broke things:

https://review.openstack.org/#/c/257744/

[Matt Riedemann (mriedem)]

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22IOError%3A%20%5BErrno%2013%5D%20Permission%20denied%3A%20'%2Fopt%2Fstack%2Fnew%2Fdevstack%2Fsrc%2Fhorizon%2Fopenstack_dashboard%2Flocal%2F_opt_stack_new_devstack_src_horizon_openstack_dashboard_local_.secret_key_store.lock'%5C%22%20AND%20voting%3A%5C%221%5C%22

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/265440

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/265440
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=08e2a666bca8c3e4926d22610346a5a992025f58
Submitter: Jenkins
Branch: master

commit 08e2a666bca8c3e4926d22610346a5a992025f58
Author: vponomaryov <email address hidden>
Date: Fri Jan 8 22:51:32 2016 +0200

    Make Manila UI be installed after Horizon

    Recent devstack change [1] made Horizon be installed at later point than it
    was. It is inconsistent with our Devstack plugin that now installs Manila UI
    earlier than Horizon. So, postpone installation of Manila UI until Horizon
    exists and fix gates by this action.

    [1] I8bf2ceaf7734c4f7cec68bc05d7cdbae81ef311e

    Change-Id: Icb2fc77235cb7f6ec4349dacddf262c3e3be76c2
    Closes-Bug: #1532300

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/manila 2.0.0.0b2 development milestone.