tox -e genconfig fails due to missing versionutils module

Bug #1469197 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Matt Riedemann

Bug Description

With this change (and probably before it actually), tox -e genconfig will fail if you don't have the oslo-config-generator's requirements in your site-packages, which is kind of dumb for a tox job:

https://review.openstack.org/#/c/180013/

mriedem@ubuntu:~/git/nova$ tox -r -e genconfig
genconfig create: /home/mriedem/git/nova/.tox/genconfig
genconfig installdeps: -r/home/mriedem/git/nova/requirements.txt, -r/home/mriedem/git/nova/test-requirements.txt
genconfig develop-inst: /home/mriedem/git/nova
genconfig installed: adium-theme-ubuntu==0.3.4,aioeventlet==0.4,alembic==0.7.6,amqp==1.4.6,anyjson==0.3.3,appdirs==1.4.0,apt-xapian-index==0.45,Babel==1.3,boto==2.38.0,cffi==1.1.2,chardet==2.0.1,cliff==1.13.0,cmd2==0.6.8,colorama==0.2.5,command-not-found==0.3,coverage==3.7.1,cryptography==0.9.1,debtagshw==0.1,debtcollector==0.5.0,decorator==3.4.2,defer==1.0.6,dirspec==13.10,discover==0.4.0,docutils==0.12,duplicity==0.6.23,ecdsa==0.13,enum34==1.0.4,eventlet==0.17.4,extras==0.0.3,fasteners==0.12.0,fixtures==1.2.0,flake8==2.2.4,functools32==3.2.3.post1,futures==3.0.3,git-review==1.24,greenlet==0.4.7,hacking==0.10.2,html5lib==0.999,httplib2==0.9.1,idna==2.0,ipaddress==1.0.7,iso8601==0.1.10,Jinja2==2.7.3,jsonpatch==1.11,jsonpointer==1.9,jsonschema==2.5.1,keyring==5.3,keystonemiddleware==2.0.0,kombu==3.0.26,launchpadlib==1.10.2,lazr.authentication==0.1.3,lazr.restfulclient==0.13.1,lazr.uri==1.0.3,linecache2==1.0.0,lockfile==0.8,lxml==3.4.4,Mako==1.0.1,MarkupSafe==0.23,mccabe==0.2.1,mock==1.0.1,monotonic==0.2,mox3==0.8.0,msgpack-python==0.4.6,MySQL-python==1.2.5,netaddr==0.7.14,netifaces==0.10.4,-e git://git.openstack.org/openstack/nova@5412ef0db46d9a483c173622b35555fc4e6f35bf#egg=nova-osee-master,numpy==1.9.2,oauth==1.0.1,oauthlib==0.6.1,oneconf==0.3.7,os-client-config==1.4.0,os-testr==0.1.0,oslo.concurrency==2.1.0,oslo.config==1.12.1,oslo.context==0.4.0,oslo.db==1.12.0,oslo.i18n==2.0.0,oslo.log==1.5.0,oslo.messaging==1.15.0,oslo.middleware==2.3.0,oslo.rootwrap==2.0.0,oslo.serialization==1.6.0,oslo.utils==1.6.0,oslo.versionedobjects==0.4.0,oslo.vmware==0.15.0,oslosphinx==3.0.0,oslotest==1.8.0,PAM==0.4.2,paramiko==1.15.2,Paste==2.0.2,PasteDeploy==1.5.2,pbr==1.2.0,pep8==1.5.7,pexpect==3.1,Pillow==2.3.0,piston-mini-client==0.7.5,pluggy==0.3.0,posix-ipc==1.0.0,prettytable==0.7.2,psutil==1.2.1,psycopg2==2.6.1,py==1.4.28,pyasn1==0.1.8,pycadf==1.0.0,pycparser==2.14,pycrypto==2.6.1,pycups==1.9.66,pycurl==7.19.3,pyflakes==0.8.1,Pygments==2.0.2,pygobject==3.12.0,pyOpenSSL==0.15.1,pyparsing==2.0.3,pyserial==2.6,pysmbc==1.0.14.1,python-apt==0.9.3.5,python-barbicanclient==3.2.0,python-cinderclient==1.2.2,python-debian===0.1.21-nmu2ubuntu2,python-glanceclient==0.19.0,python-ironicclient==0.7.0,python-keystoneclient==1.6.0,python-mimeparse==0.1.4,python-neutronclient==2.6.0,python-subunit==1.1.0,pytz==2015.4,pyxdg==0.25,PyYAML==3.11,reportlab==3.0,repoze.lru==0.6,requests==2.7.0,requests-mock==0.6.0,retrying==1.3.3,rfc3986==0.2.2,Routes==2.1,sessioninstaller==0.0.0,simplejson==3.7.3,six==1.9.0,software-center-aptd-plugins==0.0.0,Sphinx==1.2.3,SQLAlchemy==1.0.6,sqlalchemy-migrate==0.9.6,sqlparse==0.1.15,stevedore==1.5.0,suds-jurko==0.6,system-service==0.1.6,tempest-lib==0.6.1,Tempita==0.5.2,testrepository==0.0.20,testresources==0.2.7,testscenarios==0.5.0,testtools==1.8.0,tox==2.0.2,traceback2==1.4.0,trollius==1.0.4,Twisted-Core==13.2.0,Twisted-Web==13.2.0,unittest2==1.0.1,urllib3==1.10.4,virtualenv==13.0.3,wadllib==1.3.2,warlock==1.1.0,WebOb==1.4.1,websockify==0.6.1,wheel==0.24.0,wrapt==1.10.4,wsgi-intercept==0.10.0,xdiagnose===3.6.3build2,zope.interface==4.0.5
genconfig runtests: PYTHONHASHSEED='942196483'
genconfig runtests: commands[0] | oslo-config-generator --config-file=etc/nova/nova-config-generator.conf
Traceback (most recent call last):
  File ".tox/genconfig/bin/oslo-config-generator", line 11, in <module>
    sys.exit(main())
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/oslo_config/generator.py", line 262, in main
    generate(conf)
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/oslo_config/generator.py", line 226, in generate
    for namespace, listing in _list_opts(conf.namespace):
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/oslo_config/generator.py", line 201, in _list_opts
    invoke_on_load=True)
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/stevedore/named.py", line 55, in __init__
    verify_requirements)
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/stevedore/extension.py", line 170, in _load_plugins
    self._on_load_failure_callback(self, ep, err)
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins
    verify_requirements,
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/stevedore/named.py", line 123, in _load_one_plugin
    verify_requirements,
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin
    plugin = ep.resolve()
  File "/home/mriedem/git/nova/.tox/genconfig/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2361, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ImportError: No module named versionutils
ERROR: InvocationError: '/home/mriedem/git/nova/.tox/genconfig/bin/oslo-config-generator --config-file=etc/nova/nova-config-generator.conf'
___________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR: genconfig: commands failed
mriedem@ubuntu:~/git/nova$

We should set sitepackages=False in [testenv:genconfig] in tox.ini.

Tags: testing
Matt Riedemann (mriedem)
Changed in nova:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Matt Riedemann (mriedem)
Revision history for this message
Matt Riedemann (mriedem) wrote :

I guess it's actually the oslo-config-generator that's required, which comes from oslo.config, so you need to have oslo.config installed in site-packages to run the tox -e genconfig job, which kind of sucks.

Revision history for this message
Matt Riedemann (mriedem) wrote :

And you need oslo.utils for versionutils.py, which I have in my tox env:

(genconfig)mriedem@ubuntu:~/git/nova$ pip freeze | grep oslo
oslo.concurrency==2.1.0
oslo.config==1.12.1
oslo.context==0.4.0
oslo.db==1.12.0
oslo.i18n==2.0.0
oslo.log==1.5.0
oslo.messaging==1.15.0
oslo.middleware==2.3.0
oslo.rootwrap==2.0.0
oslo.serialization==1.6.0
oslo.utils==1.6.0
oslo.versionedobjects==0.4.0
oslo.vmware==0.15.0
oslosphinx==3.0.0
oslotest==1.8.0

And oslo-config-generator is in there also:

(genconfig)mriedem@ubuntu:~/git/nova$ which oslo-config-generator
/home/mriedem/git/nova/.tox/genconfig/bin/oslo-config-generator

Found the problem - it's the versionutils entry in nova-config-generator.conf, there is an entry for versionutils which doesn't exist in nova so stevedore can't load the module.

This should have been caught during code review - someone should have manually pulled down the change and tested it out since we don't have a CI job for generating and comparing nova.conf.

Changed in nova:
status: Triaged → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :
summary: - tox genconfig shouldn't rely on sitepackages
+ tox -e genconfig fails due to missing versionutils module
Changed in nova:
status: In Progress → Won't Fix
status: Won't Fix → In Progress
Matt Riedemann (mriedem)
Changed in nova:
importance: Medium → High
Changed in nova:
status: In Progress → Fix Committed
Matt Riedemann (mriedem)
Changed in nova:
milestone: none → liberty-2
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: liberty-2 → 12.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.