heat-engine throws errors when trying to import things that may not be installed

[David Moreau Simard]

Should heat-engine really throw error stacktraces when it tries to import things that may not be installed ?

For example here: http://logs.openstack.org/20/278520/1/check/gate-puppet-openstack-integration-scenario002-tempest-dsvm-centos7/6f82bf6/logs/heat/heat-engine.txt.gz
[snip]
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension [-] Could not load 'zaqar': No module named zaqarclient.queues.v1
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension [-] No module named zaqarclient.queues.v1
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension Traceback (most recent call last):
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 163, in _load_plugins
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension verify_requirements,
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension verify_requirements,
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 184, in _load_one_plugin
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension plugin = ep.resolve()
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2386, in resolve
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0)
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/zaqar.py", line 20, in <module>
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension from zaqarclient.queues.v1 import client as zaqarclient
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension ImportError: No module named zaqarclient.queues.v1
2016-02-10 17:46:07.090 21255 ERROR stevedore.extension
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension [-] Could not load 'mistral': No module named mistralclient.api
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension [-] No module named mistralclient.api
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension Traceback (most recent call last):
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 163, in _load_plugins
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension verify_requirements,
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension verify_requirements,
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 184, in _load_one_plugin
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension plugin = ep.resolve()
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2386, in resolve
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0)
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/mistral.py", line 14, in <module>
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension from mistralclient.api import base as mistral_base
2016-02-10 17:46:07.105 21255 ERROR stevedore.extension ImportError: No module named mistralclient.api
[snip]

Let's pretend I don't have zaqar or mistral in my cloud. Should I really have these errors ? Heat-engine tries to import everything, basically.

[Thomas Herve (therve)]

I don't know what's the correct behavior here. Those tracebacks are not super nice, but not really harmful either. Even if you don't have mistral in your cloud, you can have mistralclient installed and everything will be fine.

The alternative is to do the same thing as the monasca plugin, which doesn't register itself when imports fail. We'd need to define whether it's fine or not, and if we want to do this for every single client (does it make sense to do it for nova, for instance?).

[Marcus Furlong (furlongm)]

This seems to have been patched by Red Hat in OSP, but the patch hasn't landed in RDO or heat upstream:

   https://bugzilla.redhat.com/show_bug.cgi?id=1294489

Quoting...

"For completeness, the list of clients that should not be loaded is:

barbican
designate
magnum
manila
mistral
monasca
zaqar

(Some of these use conditional imports, and therefore don't generate errors.)

Of these, manila, designate and monasca define constraints that also need to be excluded."

From my own logs:

# cat /var/log/heat/heat-engine.log | grep ImportError | sed -e "s/.*Import//g" | sort -u
Error: No module named croniter
Error: No module named designateclient
Error: No module named magnumclient.openstack.common.apiclient
Error: No module named manilaclient
Error: No module named mistralclient.api
Error: No module named zaqarclient.queues.v1

Unfortunately, ROD doesn't have two of those packages, so installing all clients is not an option:

No package python-magnumclient available.
No package python-mistralclient available.

[David Moreau Simard (dmsimard)]

For the record, RDO has both python-magnumclient and python-mistralclient from Mitaka onwards.
However, it is true that these packages are not available for the stable Liberty release right now.

[Marcus Furlong (furlongm)]

monascaclient also seems to be missing:

No package python-monascaclient available.

For the RDO stable liberty release, would it be possible to patch out references to these clients?

[Jason Dunsmore (jasondunsmore)]

This was fixed https://bugs.launchpad.net/heat/+bug/1597593