Silent termination on wrong entry point from plugin

Bug #1715663 reported by George Shuklin on 2017-09-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tempest
Wishlist
Unassigned

Bug Description

I'm trying to write my own plugin for tempest. I found that if I use wrong entrypoint, then tempest will silently terminates without reporting any errors on any command (run, verify-config, etc).

Example of good entrypoint:

        [tempest.test_plugins]
        badcode=badcode2_tempest_plugin.plugin:BadCodePlugin

Example of bad entrypoint, which cause tempest to terminate silently:

        [tempest.test_plugins]
        badcode=badcode2_tempest_plugin:BadCodePlugin

(no '.plugin').

I think tempest should provide at least some information on entrypoint failure.

Matthew Treinish (treinish) wrote :

We should be logging this already: https://github.com/openstack/tempest/blob/master/tempest/test_discover/plugins.py#L190 did you check the log file?

George Shuklin (george-shuklin) wrote :

I've tried it with last tagged version 17.0.0 and I didn't check logs, but my bugreport is about cli behaviour. If entrypoint is invalid, user see nothing, application is just terminates.

p.s. I've checked this against master. It has log entry:

2017-09-07 15:12:55.716 5332 ERROR tempest ImportError: <module 'badcode2_tempest_plugin' from '/home/user/tempest/.venv/local/lib/python2.7/site-packages/badcode2_tempest_plugin-0.0.1-py2.7.egg/badcode2_tempest_plugin/__init__.pyc'> has no 'BadCodePlugin' attribute

But it still reports nothing to stdout and terminates.

chandan kumar (chkumar246) wrote :

If tempest plugin entrypoint is not properly defined, on doing tempest list-plugins will give you error. For example I have changed something in zaqar tempest plugin entrypoint. Here is the things you can get:
[centos@zaqar-devstack zaqar]$ vim zaqar.egg-info/entry_points.txt
[centos@zaqar-devstack zaqar]$ tempest list-plugins
/usr/lib/python2.7/site-packages/eventlet/convenience.py:6: DeprecationWarning: EVENTLET_NO_GREENDNS is noop, dnspython is bundled and DNS resolution is always green
  from eventlet.green import socket
/opt/stack/cinder/cinder/tests/tempest/plugin.py:19: ImportWarning: Not importing directory '/opt/stack/cinder/cinder/tests/tempest/config': missing __init__.py
  from cinder.tests.tempest import config as project_config
Could not load 'zaqar_tests': No module named tempest_plugin.plugin
No module named tempest_plugin.plugin
[centos@zaqar-devstack zaqar]$
I hope it will solve your problem.

I would not say this is a bug either. I think it would be fine to add a stdout output as well for this case.

Changed in tempest:
importance: Undecided → Wishlist
chandan kumar (chkumar246) wrote :

As from my last comment, we can see that i already logs the information about misconfiguration of the plugin. So i think it does not appears to be a bug and but marking it as wishlist if someone wants to add the stdout for the same.

tags: added: low-hanging-fruit
Changed in tempest:
status: New → Confirmed
Changed in tempest:
assignee: nobody → Manik Bindlish (manikbindlish)

I have tried to reproduce using master(Rocky):

1.) Modified "zaqar_tempest_plugin-0.0.1.dev62-py2.7.egg-info/entry_points.txt"
======================
[tempest.test_plugins]
#zaqar_tests = zaqar_tempest_plugin.plugin:ZaqarTempestPlugin
zaqar_tests = zaqar_tempest_plugin:ZaqarTempestPlugin
======================

2.) Run "tempest list-plugins"
Could not load 'zaqar_tests': 'module' object has no attribute 'ZaqarTempestPlugin'
'module' object has no attribute 'ZaqarTempestPlugin'

I have not received the traces as in comment#3 and this issue seems fixed. Please correct if I am mistaken

Instead I noticed, double time error printing of "'module' object has no attribute 'ZaqarTempestPlugin'"

Please suggest on this?

It seems tempest run is already stdout the error when there is Bad entry. Please find below the output from console. Please advice suitable action for this issue?

stack@control:~/tempest$ tempest run
No handlers could be found for logger "tempest.test_discover.plugins"

=========================
Failures during discovery
=========================
--- import errors ---
Failed to call load_tests:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/unittest2/loader.py", line 147, in loadTestsFromModule
    return load_tests(self, tests, pattern)
  File "tempest/test_discover/test_discover.py", line 26, in load_tests
    ext_plugins = plugins.TempestTestPluginManager()
  File "tempest/lib/common/utils/misc.py", line 28, in getinstance
    instances[cls] = cls()
  File "tempest/test_discover/plugins.py", line 186, in __init__
    on_load_failure_callback=self.failure_hook)
  File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 106, in __init__
    verify_requirements)
  File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 203, in _load_plugins
    self._on_load_failure_callback(self, ep, err)
  File "tempest/test_discover/plugins.py", line 191, in failure_hook
    raise err
ImportError: 'module' object has no attribute 'ZaqarTempestPlugin'

================================================================================
The above traceback was encountered during test discovery which imports all the found test modules in the specified test_path.

Changed in tempest:
assignee: Manik Bindlish (manikbindlish19) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers