Any tempest plugin import failure leads other plugin tests fails

Bug #1555524 reported by Ghanshyam Mann on 2016-03-10
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ironic
Invalid
Undecided
Unassigned
Magnum
Invalid
Critical
Unassigned
neutron
Undecided
Unassigned
tempest
Undecided
Unassigned

Bug Description

There is failure for all projects runs tempest plugin tests. That was due to fwaas tempest plugin having import error.

When testr try to list all tests and it also list the loaded plugin tests, if there is any import error in any plugin, it will leads failure to other plugin also.

I think plugin should work in isolated way, mean if Tempest detect any error from any plugin, it should just log the error and ignore that plugin tests and run others. Tempest does while registering conf options or loading plugin.

But we should have same way for import error also.

error- http://logs.openstack.org/50/289650/4/check/gate-congress-dsvm-api/6a27be7/console.html

2016-03-10 03:42:34.018 | all-plugin runtests: commands[1] | bash tools/pretty_tox.sh --concurrency=4 congress_tempest_tests
2016-03-10 03:42:36.139 | running testr
2016-03-10 03:42:39.664 | /usr/local/lib/python2.7/dist-packages/tempest_lib/__init__.py:28: DeprecationWarning: tempest-lib is deprecated for future bug-fixes and code changes in favor of tempest. Please change your imports from tempest_lib to tempest.lib
2016-03-10 03:42:39.664 | DeprecationWarning)
2016-03-10 03:42:39.916 | running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
2016-03-10 03:42:39.916 | OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
2016-03-10 03:42:39.916 | OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
2016-03-10 03:42:39.916 | OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
2016-03-10 03:42:39.917 | ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list
2016-03-10 03:42:39.917 | --- imNon-zero exit code (2) from test listing.
2016-03-10 03:42:39.949 | perror: testr failed (3)o
2016-03-10 03:42:39.950 | rt errors ---
2016-03-10 03:42:39.983 | Failed to import test module: neutron_fwaas.tests.tempest_plugin.tests.api.test_fwaas_extensions
2016-03-10 03:42:39.985 | Traceback (most recent call last):
2016-03-10 03:42:39.989 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 456, in _find_test_path
2016-03-10 03:42:39.989 | module = self._get_module_from_name(name)
2016-03-10 03:42:39.989 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 395, in _get_module_from_name
2016-03-10 03:42:39.989 | __import__(name)
2016-03-10 03:42:39.989 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/api/test_fwaas_extensions.py", line 23, in <module>
2016-03-10 03:42:39.989 | from neutron_fwaas.tests.tempest_plugin.tests.api import base
2016-03-10 03:42:39.989 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/api/base.py", line 18, in <module>
2016-03-10 03:42:39.989 | from neutron_fwaas.tests.tempest_plugin.tests import fwaas_client
2016-03-10 03:42:39.989 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/fwaas_client.py", line 25, in <module>
2016-03-10 03:42:39.990 | from neutron_fwaas.tests.tempest_plugin.services import client
2016-03-10 03:42:39.990 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/services/client.py", line 18, in <module>
2016-03-10 03:42:39.990 | from tempest.services.network.json import base
2016-03-10 03:42:39.990 | ImportError: cannot import name base
2016-03-10 03:42:39.990 |
2016-03-10 03:42:39.990 | Failed to import test module: neutron_fwaas.tests.tempest_plugin.tests.scenario.test_fwaas
2016-03-10 03:42:39.990 | Traceback (most recent call last):
2016-03-10 03:42:39.990 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 456, in _find_test_path
2016-03-10 03:42:39.990 | module = self._get_module_from_name(name)
2016-03-10 03:42:39.990 | File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 395, in _get_module_from_name
2016-03-10 03:42:39.990 | __import__(name)
2016-03-10 03:42:39.990 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/scenario/test_fwaas.py", line 21, in <module>
2016-03-10 03:42:39.991 | from neutron_fwaas.tests.tempest_plugin.tests.scenario import base
2016-03-10 03:42:39.991 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/scenario/base.py", line 21, in <module>
2016-03-10 03:42:39.991 | from neutron_fwaas.tests.tempest_plugin.tests import fwaas_client
2016-03-10 03:42:39.991 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/tests/fwaas_client.py", line 25, in <module>
2016-03-10 03:42:39.991 | from neutron_fwaas.tests.tempest_plugin.services import client
2016-03-10 03:42:39.991 | File "/opt/stack/new/neutron-fwaas/neutron_fwaas/tests/tempest_plugin/services/client.py", line 18, in <module>
2016-03-10 03:42:39.991 | from tempest.services.network.json import base
2016-03-10 03:42:39.991 | ImportError: cannot import name base
2016-03-10 03:42:39.991 | The test run didn't actually run any tests

Changed in tempest:
assignee: nobody → Ghanshyam Mann (ghanshyammann)
Ghanshyam Mann (ghanshyammann) wrote :

Actual issue is with fwaas plugin tests and fixed in https://review.openstack.org/#/c/290983/

after merging that, all tests should run fine.

hongbin (hongbin034) wrote :

This bug broke Magnum gate, so I mark it as Critical.

Changed in magnum:
importance: Undecided → Critical
hongbin (hongbin034) on 2016-03-10
Changed in magnum:
status: New → Fix Committed
Ghanshyam Mann (ghanshyammann) wrote :

Putting this for fwaas and fixed. Invalid for other projects.

Tempest we will keep and let's see if we can have nice way to handle such issues.

Changed in ironic:
status: New → Invalid
Changed in magnum:
status: Fix Committed → Invalid
Changed in neutron:
status: New → Fix Committed
Joshua White (joshua-l-white) wrote :

Set to In Progress, if you are no longer working on it please unassign and set to status accordingly.

Changed in tempest:
status: New → In Progress
chandan kumar (chkumar246) wrote :

Unassigning this bug as there is no activity from last 6 months. If you are still want to work on this bug, feel free to assign yourself.

Changed in tempest:
assignee: Ghanshyam Mann (ghanshyammann) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers