Comment 6 for bug 1263175

Revision history for this message
Abhijeet Malawade (abhijeet-malawade) wrote :

I think this issue is happening because we are stubbing 'os.path.exists' in setUp method of BaseVolumeTestCase (https://github.com/openstack/cinder/blob/master/cinder/tests/test_volume.py#L111).

Taskflow code uses stevedore module to load taskflow-engines and stevedore internally use pkg_resources.
Pkg_resources uses 'os'path.exists' to check if 'entry_pionts.txt' file is present or not. But because of stubbing 'os.path.exists' it returns
'True' every time. So even if 'entry_points.txt' is not present pkg_resources tries to open this file (https://bitbucket.org/pypa/setuptools/src/13a839bd8ad27a17543878068b996991357264d2/pkg_resources.py?at=default#cl-1454) and this call fails.

But when we run entire test suite, 'pkg_resurces' is getting called in 'cinder/openstack/common/scheduler/base_handler.py'. (base_handler uses stevedore)
This scheduler test cases are getting executed before 'test_volume' test cases. And I think 'pkg_resources' stores(caches) all distribution object
(https://bitbucket.org/pypa/setuptools/src/13a839bd8ad27a17543878068b996991357264d2/pkg_resources.py?at=default#cl-547). Because of this stevedore
loads 'taskflow.engines' successfully (gets entry points from cached distribution objects) when we run entire test suite.

If I stub pkg_resources's 'iter_entry_points' method as shown bellow then test_volume test cases are getting passed when we ran this single test module.(or entire test suite)

--- a/cinder/tests/test_volume.py
+++ b/cinder/tests/test_volume.py
@@ -29,6 +29,7 @@ import eventlet
 import mock
 import mox
 from oslo.config import cfg
+import pkg_resources
 from taskflow.engines.action_engine import engine

 from cinder.backup import driver as backup_driver
@@ -108,6 +109,30 @@ class BaseVolumeTestCase(test.TestCase):
         fake_image.stub_out_image_service(self.stubs)
         test_notifier.NOTIFICATIONS = []
         self.stubs.Set(brick_lvm.LVM, '_vg_exists', lambda x: True)
+
+ self.orignal_iter_entry_points = pkg_resources.iter_entry_points
+ def fake_iter_entry_points(group, name=None):
+ if group == 'taskflow.engines':
+ taskflow_entry_points = []
+ taskflow_entry_points.append(
+ pkg_resources.EntryPoint.parse(
+ 'default = taskflow.engines.action_engine.engine:'
+ 'SingleThreadedActionEngine'))
+ taskflow_entry_points.append(
+ pkg_resources.EntryPoint.parse(
+ 'serial = taskflow.engines.action_engine.engine:'
+ 'SingleThreadedActionEngine'))
+ taskflow_entry_points.append(
+ pkg_resources.EntryPoint.parse(
+ 'parallel = taskflow.engines.action_engine.engine:'
+ 'MultiThreadedActionEngine'))
+ return taskflow_entry_points
+ else:
+ return self.orignal_iter_entry_points(group, name=name)
+
+ # Stub iter_entry_points for taskflow
+ self.stubs.Set(pkg_resources, 'iter_entry_points',
+ fake_iter_entry_points)
         self.stubs.Set(os.path, 'exists', lambda x: True)
         self.volume.driver.set_initialized()

(I also tried to run this single module using nosetests and all tests are passing with nosetests. Nosetests also uses iter_entry_points to load 'nose.plugins' and these entry points are loaded before running test cases from test_volume module)

Let me know your thought on this.