unit test failures: Glance insist on ordereddict

[Thomas Goirand]

There's no python-ordereddict package anymore in Debian, as this is normally included in Python 2.7. I have therefore patched requirements.txt to remove ordereddict. However, even after this, I get some bad unit test errors about it. This must be fixed upstream, because there's no way (modern) downstream distributions can fix it (as the ordereddict Python package will *not* come back).

Below is the tracebacks for the 4 failed unit tests.

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_api_opts
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: Traceback (most recent call last):
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 143, in test_list_api_opts
    expected_opt_groups, expected_opt_names)
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2188, in load
    self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2202, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: ordereddict

======================================================================
FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_cache_opts
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: Traceback (most recent call last):
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 288, in test_list_cache_opts
    expected_opt_groups, expected_opt_names)
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2188, in load
    self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2202, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: ordereddict

======================================================================
FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_manage_opts
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: Traceback (most recent call last):
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 301, in test_list_manage_opts
    expected_opt_groups, expected_opt_names)
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2188, in load
    self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2202, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: ordereddict

======================================================================
FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_registry_opts
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: Traceback (most recent call last):
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 192, in test_list_registry_opts
    expected_opt_groups, expected_opt_names)
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2188, in load
    self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2202, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: ordereddict

======================================================================
FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_scrubber_opts
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: Traceback (most recent call last):
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 241, in test_list_scrubber_opts
    expected_opt_groups, expected_opt_names)
  File "/��PKGBUILDDIR��/glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2188, in load
    self.require(env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2202, in require
    items = working_set.resolve(reqs, env, installer)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
DistributionNotFound: ordereddict

[Thomas Goirand (thomas-goirand)]

Oh, and FYI, this is when building Glance Kilo RC1. It'd be nice to get this fixed before the final released. Thanks!

[Stuart McLaren (stuart-mclaren)]

With Louis' help we've reproduced this by removing ordereddict from the virtual env:

 ======================================================================
 FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_api_opts
 tags: worker-0
 ----------------------------------------------------------------------
 Traceback (most recent call last):
  File "glance/tests/unit/test_opts.py", line 143, in test_list_api_opts
    expected_opt_groups, expected_opt_names)
  File "glance/tests/unit/test_opts.py", line 45, in _test_entry_point
    list_fn = ep.load()
  File "/mnt/sam/git/glance/.tox/py27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2338, in load
    self.require(*args, **kwargs)
  File "/mnt/sam/git/glance/.tox/py27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2355, in require
    items = working_set.resolve(reqs, env, installer)
  File "/mnt/sam/git/glance/.tox/py27/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 827, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'ordereddict' distribution was not found and is required by glance-store, taskflow
 Ran 1 tests in 0.122s (-0.073s)

$ cat ./.tox/py27/lib/python2.7/site-packages/glance_store-0.4.0.dist-info/METADATA |grep order
Requires-Dist: ordereddict

$ cat ./.tox/py27/lib/python2.7/site-packages/taskflow-0.7.1.dist-info/METADATA|grep order
Requires-Dist: ordereddict

Based on their gating, taskflow still supports python 2.6:
 https://review.openstack.org/#/c/175103/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/175373

[Stuart McLaren (stuart-mclaren)]

https://review.openstack.org/175373

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/175375

[Stuart McLaren (stuart-mclaren)]

glance_store: https://review.openstack.org/175375

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/175442

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/175444

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/175373
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=20ec82f18c368bbc8a89e79d2d6eee2de2c2a9ea
Submitter: Jenkins
Branch: master

commit 20ec82f18c368bbc8a89e79d2d6eee2de2c2a9ea
Author: Stuart McLaren <email address hidden>
Date: Mon Apr 20 12:28:33 2015 +0000

    Remove ordereddict from requirements

    Ordereddict was required for py26 support. Since we have
    dropped support for py26 (we are no longer gating on it) we
    should remove it as a requirement and remove references
    to it in the code.

    Change-Id: I648edbb5d0b51c003f08f5f5c4ce700128032b38
    Partial-bug: 1445827

[Thomas Goirand (thomas-goirand)]

After applying the patch, I still get the below failures:

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_api_opts

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_registry_opts

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_scrubber_opts

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_cache_opts

FAIL: glance.tests.unit.test_opts.OptsTestCase.test_list_manage_opts

So, still 5 failures... :/ Is there the same kind of patch available for Taskflow?

[Stuart McLaren (stuart-mclaren)]

@Thomas

I've put up patches to master and stable/kilo to the glance and glance_store repos.
My understanding (I'm not a package guru) is that you will continue to see a problem
until taskflow also removes ordereddict as a dependency, so I marked this bug as
'also affects' taskflow.

There was some discussion on the list:
http://lists.openstack.org/pipermail/openstack-dev/2015-April/061963.html

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/175375
Committed: https://git.openstack.org/cgit/openstack/glance_store/commit/?id=e8884d2cb31c3364d353619a8b446ec89da547b8
Submitter: Jenkins
Branch: master

commit e8884d2cb31c3364d353619a8b446ec89da547b8
Author: Stuart McLaren <email address hidden>
Date: Mon Apr 20 12:31:34 2015 +0000

    Remove ordereddict from requirements

    Ordereddict was required for py26 support. Since we have
    dropped support for py26 (we are no longer gating on it) we
    should remove it as a requirement and remove references
    to it in the code.

    Change-Id: I5077039f95f65ece7992f14ffe14f149f73a2174
    Partial-bug: 1445827

[Robert Collins (lifeless)]

That test_opts thingm didn't we fix that at the summit when everything was exploding due to pbr 1.0 ?

[Ian Cordasco (icordasc)]

Robert, I think this pre-dates the pbr 1.0 change by a month or so.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/175442
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=a6f4a9cc504113f880e83b9193364fce55848024
Submitter: Jenkins
Branch: stable/kilo

commit a6f4a9cc504113f880e83b9193364fce55848024
Author: Stuart McLaren <email address hidden>
Date: Mon Apr 20 12:28:33 2015 +0000

    Remove ordereddict from requirements

    Ordereddict was required for py26 support. Since we have
    dropped support for py26 (we are no longer gating on it) we
    should remove it as a requirement and remove references
    to it in the code.

    Change-Id: I648edbb5d0b51c003f08f5f5c4ce700128032b38
    Partial-bug: 1445827
    (cherry picked from commit 20ec82f18c368bbc8a89e79d2d6eee2de2c2a9ea)

[OpenStack Infra (hudson-openstack)]

Change abandoned by Joshua Harlow (<email address hidden>) on branch: master
Review: https://review.openstack.org/187391