multiprocessing race condition when opening filesystem store

[Mark Washenberger]

When running glance in multiprocessing mode, there is a race for api processes to create the filesystem store location if it doesn't already exist. This causes only a minor problem as the wsgi server will restart the dead child process. However, it contributes heavily to the fragility of some of the functional tests.

Here is a sample trace:

2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/.venv/local/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call
2012-10-13 22:45:02 29777 TRACE glance val = callable(*args, **kw)
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/common/wsgi.py", line 408, in factory
2012-10-13 22:45:02 29777 TRACE glance return cls(routes.Mapper())
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/api/v2/router.py", line 42, in __init__
2012-10-13 22:45:02 29777 TRACE glance images_resource = images.create_resource(custom_image_properties)
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/api/v2/images.py", line 726, in create_resource
2012-10-13 22:45:02 29777 TRACE glance controller = ImagesController()
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/api/v2/images.py", line 51, in __init__
2012-10-13 22:45:02 29777 TRACE glance self.store_api.create_stores()
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/store/__init__.py", line 167, in create_stores
2012-10-13 22:45:02 29777 TRACE glance store_instance = store_cls()
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/store/base.py", line 41, in __init__
2012-10-13 22:45:02 29777 TRACE glance self.configure_add()
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/glance/store/filesystem.py", line 128, in configure_add
2012-10-13 22:45:02 29777 TRACE glance os.makedirs(self.datadir)
2012-10-13 22:45:02 29777 TRACE glance File "/home/markwash/devel/glance/.venv/lib/python2.7/os.py", line 157, in makedirs
2012-10-13 22:45:02 29777 TRACE glance mkdir(name, mode)
2012-10-13 22:45:02 29777 TRACE glance OSError: [Errno 17] File exists: '/tmp/test.72622/images'
2012-10-13 22:45:02 29777 TRACE glance

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/14441

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/14441
Committed: http://github.com/openstack/glance/commit/04b0ed7ae4966259aa1967dc7e5ebb093be583c8
Submitter: Jenkins
Branch: master

commit 04b0ed7ae4966259aa1967dc7e5ebb093be583c8
Author: Mark Washenberger <email address hidden>
Date: Sun Oct 14 00:03:26 2012 +0000

    Fix glance filesystem store race condition

    Fixes bug 1066416

    Change-Id: I94c57b79efa251c5e11643e13052a549ab487dae