Unit tests failing on vine.five import

Bug #1638263 reported by David McNally
30
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Ceilometer
Invalid
Undecided
Lianhao Lu
OpenStack Global Requirements
Fix Released
Medium
Unassigned
keystonemiddleware
Invalid
Undecided
Unassigned
oslo.messaging
Fix Released
Undecided
Gevorg Davoian

Bug Description

When running the oslo.messaging unit tests they fail with the following error:

$ tox -e py27
GLOB sdist-make: oslo.messaging/setup.py
py27 create: oslo.messaging/.tox/py27
py27 installdeps: -r oslo.messaging/test-requirements.txt
py27 inst: oslo.messaging/.tox/dist/oslo.messaging-5.11.1.dev18.zip
py27 installed: alabaster==0.7.9,amqp==1.4.9,appdirs==1.4.0,Babel==2.3.4,bandit==1.1.0,cachetools==2.0.0,cffi==1.8.3,cliff==2.2.0,cmd2==0.6.9,contextlib2==0.5.4,coverage==4.2,debtcollector==1.9.0,docutils==0.12,enum34==1.1.6,eventlet==0.19.0,extras==1.0.0,fasteners==0.14.1,fixtures==3.0.0,flake8==2.2.4,funcsigs==1.0.2,futures==3.0.5,futurist==0.19.0,gitdb2==2.0.0,GitPython==2.1.0,greenlet==0.4.10,hacking==0.10.2,iso8601==0.1.11,Jinja2==2.8,kafka-python==0.9.5,keystoneauth1==2.14.0,kombu==4.0.0,linecache2==1.0.0,MarkupSafe==0.23,mccabe==0.2.1,mock==2.0.0,monotonic==1.2,mox3==0.18.0,msgpack-python==0.4.8,netaddr==0.7.18,netifaces==0.10.5,os-client-config==1.22.0,oslo.concurrency==3.15.0,oslo.config==3.17.0,oslo.context==2.10.0,oslo.i18n==3.10.0,oslo.log==3.17.0,oslo.messaging==5.11.1.dev18,oslo.middleware==3.20.0,oslo.serialization==2.14.0,oslo.service==1.17.0,oslo.utils==3.17.0,oslosphinx==4.8.0,oslotest==2.11.0,Paste==2.0.3,PasteDeploy==1.5.2,pbr==1.10.0,pep8==1.5.7,pifpaf==0.13.0,pika==0.10.0,pika-pool==0.1.3,positional==1.1.1,prettytable==0.7.2,pycparser==2.17,pyflakes==0.8.1,Pygments==2.1.3,pyinotify==0.9.6,pyngus==2.1.2,pyparsing==2.1.10,python-dateutil==2.5.3,python-mimeparse==1.6.0,python-qpid-proton==0.14.0,python-subunit==1.2.0,pytz==2016.7,PyYAML==3.12,pyzmq==16.0.0,redis==2.10.5,reno==1.8.0,repoze.lru==0.6,requests==2.11.1,requestsexceptions==1.1.3,retrying==1.3.3,rfc3986==0.4.1,Routes==2.3.1,six==1.10.0,smmap2==2.0.1,snowballstemmer==1.2.1,Sphinx==1.3.6,sphinx-rtd-theme==0.1.9,stevedore==1.18.0,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.2.0,traceback2==1.4.0,unicodecsv==0.14.1,unittest2==1.1.0,WebOb==1.6.2,wrapt==1.10.8,xattr==0.9.1
py27 runtests: PYTHONHASHSEED='347742952'
py27 runtests: commands[0] | python setup.py testr --slowest --testr-args=
running testr
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} ${PYTHON:-python} -m subunit.run discover -t ./ . --list
No handlers could be found for logger "kafka.conn"
--- import errors ---
Failed to import test module: oslo_messaging.tests.drivers.test_impl_rabbit
Traceback (most recent call last):
  File "oslo.messaging/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "oslo.messaging/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "oslo_messaging/tests/drivers/test_impl_rabbit.py", line 24, in <module>
    import kombu.transport.memory
  File "oslo.messaging/.tox/py27/local/lib/python2.7/site-packages/kombu/transport/__init__.py", line 4, in <module>
    from kombu.five import string_t
  File "oslo.messaging/.tox/py27/local/lib/python2.7/site-packages/kombu/five.py", line 6, in <module>
    import vine.five
ImportError: No module named vine.five
Non-zero exit code (2) from test listing.
error: testr failed (3)
ERROR: InvocationError: 'oslo.messaging/.tox/py27/bin/python setup.py testr --slowest --testr-args='

Revision history for this message
David McNally (dave-mcnally) wrote :

This error happens when running the tests with kombu version 4.0.0. When I restrict the version to 3.0.33 the test run completes successfully.

affects: oslo.messaging → kombu
affects: kombu → oslo.messaging
Changed in oslo.messaging:
assignee: nobody → Gevorg Davoian (gdavoian)
Changed in keystonemiddleware:
assignee: nobody → Gevorg Davoian (gdavoian)
Revision history for this message
Tony Breeds (o-tony) wrote :

Added OpenStack Global Requirements to track this issue as the fix is likely to to have largish impacts

Changed in openstack-requirements:
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
Matthew Thode (prometheanfire) wrote :

from what was communicated in https://review.openstack.org/392135 upstream considers this to be our problem, meaning we need to work around it. The obvious immediate solution is to cap kombu at <4. I am not aware of a better solution on our side but would like one, because even if we do cap it, we will eventually need to uncap it.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to oslo.messaging (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/392354

Revision history for this message
Tony Breeds (o-tony) wrote :

From a openstack/requirements POV we have a couple of options

https://review.openstack.org/#/c/392245/ (uncap amqp)
https://review.openstack.org/#/c/392135/ (cap kombu)

The first option is better as we're trying hard to *NOT* have caps where we can and it removes one :) However amqp was capped in https://review.openstack.org/#/c/322484/ because kombu itself required amqp < 2.0 https://github.com/celery/kombu/blob/3.0/requirements/default.txt

With the release of 4.0 we can see that the inverse is now true: https://github.com/celery/kombu/commit/b48a887a865971f98f3913c24daeb64e9db52527

So it seems to me that altering the cap on amqp is the correct solution here.

I've uploaded https://review.openstack.org/392354 to look into this

Revision history for this message
Matthew Thode (prometheanfire) wrote :

oh, that would be nice, will look into that once tests pass

Changed in oslo.messaging:
status: New → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on oslo.messaging (master)

Change abandoned by Tony Breeds (<email address hidden>) on branch: master
Review: https://review.openstack.org/392354

Revision history for this message
Dirk Mueller (dmllr) wrote :

Why is oslo not using upper constraints?

Revision history for this message
Jerry Zhao (zhaoxinyu) wrote :

kombu 4.0.1 is released today and still has that issue.

Lianhao Lu (lianhao-lu)
Changed in ceilometer:
assignee: nobody → Lianhao Lu (lianhao-lu)
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/408934

Changed in ceilometer:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/409031

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on ceilometer (master)

Change abandoned by Lianhao Lu (<email address hidden>) on branch: master
Review: https://review.openstack.org/409031
Reason: abandon this

Mehdi Abaakouk (sileht)
Changed in ceilometer:
status: In Progress → Invalid
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Julien Danjou (<email address hidden>) on branch: master
Review: https://review.openstack.org/408934

Revision history for this message
Steve Martinelli (stevemar) wrote :

Ran py27 on master, newton and mitaka. everything seems fine.

Changed in keystonemiddleware:
status: New → Invalid
assignee: Gevorg Davoian (gdavoian) → nobody
no longer affects: watcher
Revision history for this message
Lianhao Lu (lianhao-lu) wrote :

with kombu 4.0.2 released, this issue appeared again. We need to wait for https://review.openstack.org/#/c/411716 merged and new oslo.messaging release.

I'm thinking we should add a temporary cap on the kombu version before we resolve the kombu version 4 support issue.

Revision history for this message
Matthew Thode (prometheanfire) wrote :

closing as this has been fixed for a while.

Changed in openstack-requirements:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.