list-verifier-tests fails when barbican-tempest-plugin is added

[Tatsuma Matsuki]

If I try to get the test list by running "rally verify list-verifier-tests",
It fails especially when I add barbican-tempest-plugin extension.

As far as I investigated, the command does not load the appropriate
"auth" config in tempest.conf.
To solve this problem, "verifier.set_deployment()" need to be called to set
TEMPEST_CONFIG and TEMPEST_CONFIG_DIR.

Or, is this a bug on barbican-tempest-plugin?

To reproduce this error, run the following commands.

$ rally verify create-verifier --type tempest --name tempest-verifier --source https://github.com/openstack/tempest.git
$ rally verify add-verifier-ext --source https://github.com/openstack/barbican-tempest-plugin.git
$ rally verify list-verifier-tests

Error messages:

Failed to import test module: barbican_tempest_plugin.tests.scenario.test_volume_encryption
Traceback (most recent call last):
  File "/root/.rally/verification/verifier-04b6fce4-d378-4ec8-99d9-d1a909b5af19/.venv/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/root/.rally/verification/verifier-04b6fce4-d378-4ec8-99d9-d1a909b5af19/.venv/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "/root/.rally/verification/verifier-04b6fce4-d378-4ec8-99d9-d1a909b5af19/extensions/barbican-tempest-plugin/barbican_tempest_plugin/tests/scenario/test_volume_encryption.py", line 20, in <module>
    from barbican_tempest_plugin.tests.scenario import barbican_manager
  File "/root/.rally/verification/verifier-04b6fce4-d378-4ec8-99d9-d1a909b5af19/extensions/barbican-tempest-plugin/barbican_tempest_plugin/tests/scenario/barbican_manager.py", line 32, in <module>
    from barbican_tempest_plugin import clients
  File "/root/.rally/verification/verifier-04b6fce4-d378-4ec8-99d9-d1a909b5af19/extensions/barbican-tempest-plugin/barbican_tempest_plugin/clients.py", line 22, in <module>
    ADMIN_CREDS = common_creds.get_configured_admin_credentials()
  File "tempest/common/credentials_factory.py", line 261, in get_configured_admin_credentials
    identity_version=identity_version, **params)
  File "tempest/common/credentials_factory.py", line 305, in get_credentials
    **params)
  File "tempest/lib/auth.py", line 644, in get_credentials
    http_timeout=http_timeout)
  File "tempest/lib/auth.py", line 265, in __init__
    super(KeystoneAuthProvider, self).__init__(credentials, scope)
  File "tempest/lib/auth.py", line 93, in __init__
    raise exceptions.InvalidCredentials(message)
InvalidCredentials: Invalid Credentials
Details: Credentials are: {'username': None, 'project_name': None, 'project_domain_id': None, 'user_domain_id': None, 'tenant_id': None, 'user_domain_name': None, 'domain_name': None, 'tenant_name': None, 'user_id': None, 'project_id': None, 'domain_id': None, 'project_domain_name': None} Password is not defined.
Non-zero exit code (2) from test listing.
': CalledProcessError: Command '['testr', 'list-tests', '']' returned non-zero exit status 3
Command failed, please check log for more info
2017-10-13 09:22:54.761 13144 CRITICAL rally [-] Unhandled error: CalledProcessError: Command '['testr', 'list-tests', '']' returned non-zero exit status 3
2017-10-13 09:22:54.761 13144 ERROR rally Traceback (most recent call last):
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/bin/rally", line 10, in <module>
2017-10-13 09:22:54.761 13144 ERROR rally sys.exit(main())
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/main.py", line 40, in main
2017-10-13 09:22:54.761 13144 ERROR rally return cliutils.run(sys.argv, categories)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/cliutils.py", line 659, in run
2017-10-13 09:22:54.761 13144 ERROR rally ret = fn(*fn_args, **fn_kwargs)
2017-10-13 09:22:54.761 13144 ERROR rally File "<decorator-gen-60>", line 2, in list_verifier_tests
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/envutils.py", line 67, in default_from_global
2017-10-13 09:22:54.761 13144 ERROR rally return f(*args, **kwargs)
2017-10-13 09:22:54.761 13144 ERROR rally File "<decorator-gen-59>", line 2, in list_verifier_tests
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/plugins/__init__.py", line 51, in ensure_plugins_are_loaded
2017-10-13 09:22:54.761 13144 ERROR rally return f(*args, **kwargs)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/cli/commands/verify.py", line 366, in list_verifier_tests
2017-10-13 09:22:54.761 13144 ERROR rally pattern=pattern)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/api.py", line 964, in list_tests
2017-10-13 09:22:54.761 13144 ERROR rally return verifier.manager.list_tests(pattern)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/verification/tempest/manager.py", line 194, in list_tests
2017-10-13 09:22:54.761 13144 ERROR rally return super(TempestManager, self).list_tests(pattern)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/plugins/common/verification/testr.py", line 110, in list_tests
2017-10-13 09:22:54.761 13144 ERROR rally debug_output=False)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/local/lib/python2.7/dist-packages/rally/verification/utils.py", line 53, in check_output
2017-10-13 09:22:54.761 13144 ERROR rally output = subprocess.check_output(*args, **kwargs)
2017-10-13 09:22:54.761 13144 ERROR rally File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
2017-10-13 09:22:54.761 13144 ERROR rally raise CalledProcessError(retcode, cmd, output=output)
2017-10-13 09:22:54.761 13144 ERROR rally CalledProcessError: Command '['testr', 'list-tests', '']' returned non-zero exit status 3
2017-10-13 09:22:54.761 13144 ERROR rally

[Sofiia Andriichenko (sandriichenko)]

I had those problem, because rally try to use /etc/tempest/tempest.config(it doesn't exist) instead of created config by rally

my workaround: https://github.com/sandriichenko/rally_tempest_docker/commit/9db41bf1fcdf849955deabb1f52cb94e275fa671#diff-33d4e96608fd2cc7b5413e4eb3f48fce

[Anandprakash (atandale)]

I could see a similar problem but not the exact one.
When I was trying to run a command - 'tempest run -l', I could see the following error-

--- import errors ---\nFailed to import test module: barbican_tempest_plugin.tests.api.test_consumers\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/unittest2/loader.py", line 456, in _find_test_path\n module = self._get_module_from_name(name)\n File "/usr/lib/python3.6/site-packages/unittest2/loader.py", line 395, in _get_module_from_name\n __import__(name)\n File "/usr/lib/python3.6/site-packages/barbican_tempest_plugin/tests/api/test_consumers.py",