Taskflow code uses stevedore module to load taskflow-engines and stevedore internally use pkg_resources.
Pkg_resources uses 'os'path.exists' to check if 'entry_pionts.txt' file is present or not. But because of stubbing 'os.path.exists' it returns
'True' every time. So even if 'entry_points.txt' is not present pkg_resources tries to open this file (https://bitbucket.org/pypa/setuptools/src/13a839bd8ad27a17543878068b996991357264d2/pkg_resources.py?at=default#cl-1454) and this call fails.
But when we run entire test suite, 'pkg_resurces' is getting called in 'cinder/openstack/common/scheduler/base_handler.py'. (base_handler uses stevedore)
This scheduler test cases are getting executed before 'test_volume' test cases. And I think 'pkg_resources' stores(caches) all distribution object
(https://bitbucket.org/pypa/setuptools/src/13a839bd8ad27a17543878068b996991357264d2/pkg_resources.py?at=default#cl-547). Because of this stevedore
loads 'taskflow.engines' successfully (gets entry points from cached distribution objects) when we run entire test suite.
If I stub pkg_resources's 'iter_entry_points' method as shown bellow then test_volume test cases are getting passed when we ran this single test module.(or entire test suite)
--- a/cinder/tests/test_volume.py
+++ b/cinder/tests/test_volume.py
@@ -29,6 +29,7 @@ import eventlet
import mock
import mox
from oslo.config import cfg
+import pkg_resources
from taskflow.engines.action_engine import engine
(I also tried to run this single module using nosetests and all tests are passing with nosetests. Nosetests also uses iter_entry_points to load 'nose.plugins' and these entry points are loaded before running test cases from test_volume module)
I think this issue is happening because we are stubbing 'os.path.exists' in setUp method of BaseVolumeTestCase (https:/ /github. com/openstack/ cinder/ blob/master/ cinder/ tests/test_ volume. py#L111).
Taskflow code uses stevedore module to load taskflow-engines and stevedore internally use pkg_resources. /bitbucket. org/pypa/ setuptools/ src/13a839bd8ad 27a17543878068b 996991357264d2/ pkg_resources. py?at=default# cl-1454) and this call fails.
Pkg_resources uses 'os'path.exists' to check if 'entry_pionts.txt' file is present or not. But because of stubbing 'os.path.exists' it returns
'True' every time. So even if 'entry_points.txt' is not present pkg_resources tries to open this file (https:/
But when we run entire test suite, 'pkg_resurces' is getting called in 'cinder/ openstack/ common/ scheduler/ base_handler. py'. (base_handler uses stevedore) /bitbucket. org/pypa/ setuptools/ src/13a839bd8ad 27a17543878068b 996991357264d2/ pkg_resources. py?at=default# cl-547). Because of this stevedore
This scheduler test cases are getting executed before 'test_volume' test cases. And I think 'pkg_resources' stores(caches) all distribution object
(https:/
loads 'taskflow.engines' successfully (gets entry points from cached distribution objects) when we run entire test suite.
If I stub pkg_resources's 'iter_entry_points' method as shown bellow then test_volume test cases are getting passed when we ran this single test module.(or entire test suite)
--- a/cinder/ tests/test_ volume. py tests/test_ volume. py engines. action_ engine import engine
+++ b/cinder/
@@ -29,6 +29,7 @@ import eventlet
import mock
import mox
from oslo.config import cfg
+import pkg_resources
from taskflow.
from cinder.backup import driver as backup_driver ase(test. TestCase) :
fake_ image.stub_ out_image_ service( self.stubs)
test_ notifier. NOTIFICATIONS = []
self. stubs.Set( brick_lvm. LVM, '_vg_exists', lambda x: True) iter_entry_ points = pkg_resources. iter_entry_ points entry_points( group, name=None): entry_points = [] entry_points. append( EntryPoint. parse( engines. action_ engine. engine: ' ActionEngine' )) entry_points. append( EntryPoint. parse( engines. action_ engine. engine: ' ActionEngine' )) entry_points. append( EntryPoint. parse( engines. action_ engine. engine: ' ctionEngine' )) entry_points iter_entry_ points( group, name=name) Set(pkg_ resources, 'iter_entry_ points' , entry_points)
self. stubs.Set( os.path, 'exists', lambda x: True)
self. volume. driver. set_initialized ()
@@ -108,6 +109,30 @@ class BaseVolumeTestC
+
+ self.orignal_
+ def fake_iter_
+ if group == 'taskflow.engines':
+ taskflow_
+ taskflow_
+ pkg_resources.
+ 'default = taskflow.
+ 'SingleThreaded
+ taskflow_
+ pkg_resources.
+ 'serial = taskflow.
+ 'SingleThreaded
+ taskflow_
+ pkg_resources.
+ 'parallel = taskflow.
+ 'MultiThreadedA
+ return taskflow_
+ else:
+ return self.orignal_
+
+ # Stub iter_entry_points for taskflow
+ self.stubs.
+ fake_iter_
(I also tried to run this single module using nosetests and all tests are passing with nosetests. Nosetests also uses iter_entry_points to load 'nose.plugins' and these entry points are loaded before running test cases from test_volume module)
Let me know your thought on this.