can't run individual tests anymore (again)

[Michael Kerrin]

sgh I can't run individual tests anymore.

Bug #1182037 and its implementation https://review.openstack.org/#/c/33030/ have the side effect of importing the cinder.common.config module only in the bin/ scripts or in the cinder.tests module. So when I try and run individual tests I exercise a different import ordering and get the following error:

raceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 368, in <module>
    main(sys.argv, sys.stdout)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 365, in main
    stdout=stdout)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 175, in __init__
    self.parseArgs(argv)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 252, in parseArgs
    self.createTests()
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 261, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "cinder/volume/__init__.py", line 30, in <module>
    API = import_utils.import_class(CONF.volume_api_class)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1187, in __getattr__
    return self._get(name)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1509, in _get
    value = self._substitute(self._do_get(name, group))
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1525, in _do_get
    info = self._get_opt_info(name, group)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1625, in _get_opt_info
    raise NoSuchOptError(opt_name, group)
oslo.config.cfg.NoSuchOptError: no such option: volume_api_class

More worringly on a clean branch I am getting the following failing test, due to the same issue. I am wondering why it is just me getting this and the jenkins gate job is failing on this?

======================================================================
FAIL: unittest.loader.ModuleImportFailure.cinder.tests.test_backup_ceph
----------------------------------------------------------------------
_StringException: ImportError: Failed to import test module: cinder.tests.test_backup_ceph
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 230, in _get_module_from_name
    __import__(name)
  File "/opt/stack/cinder/cinder/tests/test_backup_ceph.py", line 22, in <module>
    from cinder.backup.drivers.ceph import CephBackupDriver
  File "/opt/stack/cinder/cinder/backup/__init__.py", line 27, in <module>
    API = cinder.openstack.common.importutils.import_class(CONF.backup_api_class)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1187, in __getattr__
    return self._get(name)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1509, in _get
    value = self._substitute(self._do_get(name, group))
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1525, in _do_get
    info = self._get_opt_info(name, group)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1625, in _get_opt_info
    raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: backup_api_class

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/38148

[Xiaoxi Chen (xiaoxi-chen)]

Well, I can run the individual test, is it your configuration error ?

root@ceph-test:/opt/stack/cinder# ./run_tests.sh test_backup_ceph
Running `tools/with_venv.sh python setup.py testr --testr-args='--subunit --concurrency 1 test_backup_ceph'`
cinder.tests.test_backup_ceph.BackupCephTestCase
    test_backup_error1 OK 0.22
    test_backup_error2 OK 0.12
    test_backup_good OK 0.06
    test_backup_volume_from_file OK 0.03
    test_delete OK 0.06
    test_get_rbd_support OK 0.03
    test_restore OK 0.09
    test_tranfer_data_from_rbd OK 0.03
    test_tranfer_data_to_rbd OK 0.03

Ran 9 tests in 4.755s

OK

[Michael Kerrin (michael-kerrin-w)]

kerrin@devstack:/opt/stack/cinder$ git status
# On branch master
nothing to commit (working directory clean)

kerrin@devstack:/opt/stack/cinder$ ./run_tests.sh
................
======================================================================
FAIL: unittest.loader.ModuleImportFailure.cinder.tests.test_backup_ceph
----------------------------------------------------------------------
_StringException: ImportError: Failed to import test module: cinder.tests.test_backup_ceph
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 230, in _get_module_from_name
    __import__(name)
  File "/opt/stack/cinder/cinder/tests/test_backup_ceph.py", line 24, in <module>
    from cinder.backup.drivers import ceph
  File "/opt/stack/cinder/cinder/backup/__init__.py", line 27, in <module>
    API = cinder.openstack.common.importutils.import_class(CONF.backup_api_class)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1187, in __getattr__
    return self._get(name)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1509, in _get
    value = self._substitute(self._do_get(name, group))
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1525, in _do_get
    info = self._get_opt_info(name, group)
  File "/opt/stack/cinder/.venv/local/lib/python2.7/site-packages/oslo/config/cfg.py", line 1625, in _get_opt_info
    raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: backup_api_class

Ran 1637 tests in 68.096s

FAILED (failures=1)

kerrin@devstack:/opt/stack/cinder$ ./run_tests.sh -d cinder.tests.test_volume.VolumeTestCase
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 368, in <module>
    main(sys.argv, sys.stdout)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 365, in main
    stdout=stdout)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 175, in __init__
    self.parseArgs(argv)
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 252, in parseArgs
    self.createTests()
  File "/opt/stack/cinder/.venv/lib/python2.7/site-packages/testtools/run.py", line 261, in createTests
    self.module)
  File "/usr/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "cinder/tests/test_volume.py", line 32, in <module>
    from cinder.brick.iscsi import iscsi
  File "cinder/brick/iscsi/iscsi.py", line 33, in <module>
    from cinder.volume import utils as volume_utils
  File "cinder/volume/__init__....

[Huang Zhiteng (zhiteng-huang)]

Unittests (all or single) works pretty well for me. Can you fetch the latest code in trunk and try again?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/38148
Committed: http://github.com/openstack/cinder/commit/546ed2dc498e1182859ebab19a3459e49bbe48ca
Submitter: Jenkins
Branch: master

commit 546ed2dc498e1182859ebab19a3459e49bbe48ca
Author: Michael Kerrin <email address hidden>
Date: Mon Jul 22 13:35:37 2013 +0000

    Fix oslo.config.cfg.NoSuchOptError when running individual tests

    We need to import cinder.common.config wherever we are using an olso
    configuration option from this module. Otherwise it is not necessarily
    the case that olso knows about the option and it complains.

    cinder.tests imports this configuration module but it not the case that
    this module has been imported when you you are running individual tests.

    Change-Id: I8f2c966848f55c80213d2ecd2dbf75b777627e6b
    Fixes: bug #1203737