kombu tests incompatible with anyjson-0.2.4

[Eugene Kirpichov]

openstack@openstack-dev:~/openstack-master/openstack-common$ pip freeze | awk '/kombu/||/anyjson/'
anyjson==0.2.4
kombu==2.1.8

openstack@openstack-dev:~/openstack-master/openstack-common$ nosetests
...
Test kombu rpc.cast ... SKIP: Test requires kombu
test_cast_success (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: rpc driver not available.
Test kombu rpc.cast ... SKIP: Test requires kombu
test_declare_consumer_errors_will_reconnect (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: Test requires kombu
Test that an IOError exception causes a reconnection ... SKIP: Test requires kombu
Test sending to a direct exchange/queue ... SKIP: Test requires kombu
Test sending to a fanout exchange and consuming from 2 queues ... SKIP: kombu memory transport seems buggy with fanout queues as this test passes when you use rabbit (fake_rabbit=False)
test_fanout_success (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: kombu memory transport seems buggy with fanout queues as this test passes when you use rabbit (fake_rabbit=False)
test_iterconsume_errors_will_reconnect (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: Test requires kombu
test_publishing_errors_will_reconnect (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: Test requires kombu
test_queue_declared_ha_if_ha_on (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: Test requires kombu
test_queue_not_declared_ha_if_ha_off (tests.unit.rpc.test_kombu.RpcKombuTestCase) ... SKIP: Test requires kombu
Test that reusing a connection returns same one. ... SKIP: Test requires kombu
Test that rabbits are tried in roundrobin at connection failures. ... SKIP: Test requires kombu
Test sending to a topic exchange with multiple queues ... SKIP: Test requires kombu
Test sending to a topic exchange/queue ... SKIP: Test requires kombu
test_ssl_on_extended (tests.unit.rpc.test_kombu_ssl.RpcKombuSslTestCase) ... SKIP: Test requires kombu
...

If we dig deeper into this (by adding "traceback.print_exc()" to the "except ImportError" section in tests/unit/rpc/test_kombu.py), we'll find this:

Traceback (most recent call last):
  File "/home/openstack/openstack-master/openstack-common/tests/unit/rpc/test_kombu.py", line 41, in <module>
    from openstack.common.rpc import impl_kombu
  File "/home/openstack/openstack-master/openstack-common/openstack/common/rpc/impl_kombu.py", line 30, in <module>
    import kombu.messaging
  File "/usr/local/lib/python2.7/dist-packages/kombu/messaging.py", line 17, in <module>
    from .serialization import encode
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 329, in <module>
    register_json()
  File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line 267, in register_json
    from anyjson import loads, dumps
ImportError: cannot import name loads

All works fine if you install anyjson-0.3.3.

The root cause of this change in anyjson: https://bitbucket.org/runeh/anyjson/changeset/4e162c45cc7c .
As we can see from https://bitbucket.org/runeh/anyjson/changesets , the first version with this change is 0.3.2.

Root cause in kombu: https://github.com/celery/kombu/commit/ee7b17f2060f6955f86781cf9d60e19020d07016
The earliest tag of kombu which requires this change in anyjson is 2.0.0 (according to git tag --contains ee7b17f2).

So, we need EITHER (kombu < 2.0.0 AND anyjson < 0.3.2) OR (kombu >= 2.0.0 AND anyjson >= 0.3.2).

I suggest using the more up-to-date packages - kombu >= 2.0.0 and anyjson >= 0.3.2 (in which case why not just use their latest versions).

[Eugene Kirpichov (ekirpichov)]

Correction: Tests work fine with anyjson-0.2.4 and kombu-1.0.4; anyjson-0.3.3 is compatible with BOTH versions of kombu.

So, it's better to use anyjson-0.3.3 as it will ease the upgrade path for kombu.

[Mark McLoughlin (markmc)]

> Correction: Tests work fine with anyjson-0.2.4 and kombu-1.0.4

Ok, sounds to me like the bug is best marked as Invalid

We will need to update pip-requires to newer anyjson when we update to newer kombu, but not before