pkgutil.walk_packages will walk top-level packages

[Chris Dent]

The automatic import routine in os-traits can bust out of the current package and start importing other python packages, leading to errors like http://logs.openstack.org/78/469578/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/8eead8d/logs/devstacklog.txt.gz#_2017-05-31_17_48_49_751 (a snippet of which is below).

This is because walk_packages really does walk all packages. It can be constrained from doing so with a prefix. This will break anything trying to use the current release.

2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/bin/nova-manage", line 10, in <module>
2017-05-31 17:48:49.751 | ERROR nova  sys.exit(main())
2017-05-31 17:48:49.751 | ERROR nova  File "/opt/stack/new/nova/nova/cmd/manage.py", line 1667, in main
2017-05-31 17:48:49.751 | ERROR nova  objects.register_all()
2017-05-31 17:48:49.751 | ERROR nova  File "/opt/stack/new/nova/nova/objects/__init__.py", line 64, in register_all
2017-05-31 17:48:49.751 | ERROR nova  __import__('nova.objects.resource_provider')
2017-05-31 17:48:49.751 | ERROR nova  File "/opt/stack/new/nova/nova/objects/resource_provider.py", line 18, in <module>
2017-05-31 17:48:49.751 | ERROR nova  import os_traits
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/os_traits/__init__.py", line 65, in <module>
2017-05-31 17:48:49.751 | ERROR nova  import_submodules(sys.modules.get(__name__))
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/os_traits/__init__.py", line 52, in import_submodules
2017-05-31 17:48:49.751 | ERROR nova  for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/lib/python2.7/pkgutil.py", line 125, in walk_packages
2017-05-31 17:48:49.751 | ERROR nova  for item in walk_packages(path, name+'.', onerror):
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/lib/python2.7/pkgutil.py", line 125, in walk_packages
2017-05-31 17:48:49.751 | ERROR nova  for item in walk_packages(path, name+'.', onerror):
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/lib/python2.7/pkgutil.py", line 110, in walk_packages
2017-05-31 17:48:49.751 | ERROR nova  __import__(name)
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/tests/contrib/django_util/__init__.py", line 22, in <module>
2017-05-31 17:48:49.751 | ERROR nova  django.setup()
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 17, in setup
2017-05-31 17:48:49.751 | ERROR nova  configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
2017-05-31 17:48:49.751 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 48, in __getattr__
2017-05-31 17:48:49.751 | ERROR nova  self._setup(name)
2017-05-31 17:48:49.752 | ERROR nova  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 42, in _setup
2017-05-31 17:48:49.752 | ERROR nova  % (desc, ENVIRONMENT_VARIABLE))
2017-05-31 17:48:49.752 | ERROR nova ImproperlyConfigured: Requested setting LOGGING_CONFIG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
2017-05-31 17:48:49.752 | ERROR nova 

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/469625

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/469625
Committed: https://git.openstack.org/cgit/openstack/os-traits/commit/?id=789ce955eac04adb71ec5f2bc4890dd8c86c7ceb
Submitter: Jenkins
Branch: master

commit 789ce955eac04adb71ec5f2bc4890dd8c86c7ceb
Author: Chris Dent <email address hidden>
Date: Wed May 31 18:56:44 2017 +0000

    Correctly recurse os_traits packages

    When walk_packages is called, unless a prefix is provided, it is
    easy to escape the os_traits package and start walking other
    packages. If there is a top-level 'tests' package somewhere in the
    python path, this can then lead to importing all sorts of things,
    some which rely on the environment having certain characteristics
    and fail the import without that, some which will change the
    environment.

    Neither of these things are good. Nor do we want to be walking
    packages all over the place. By providing a prefix we stay within
    the os_traits package as desired.

    Change-Id: Icde41191e39317ff5a031fff9ece0f05f1e0b4bc
    Closes-Bug: #1694792

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/os-traits 0.3.1 release.