fixtures 1.2.0 isn't compatible with testtools 2.0.0

[Tony Breeds]

Note this isn't strictly speaking a fixtures but but it can be fixed readily there. testtools 2.0.0 requires fixtures >= 1.3.0 however the requirement from fixtures (in 1.2.0) on testtools is >0.9.22.

The can lead to a situation (from openstack) where we install testtools 1.9.0 and then install -U fixtures==1.2.0 and end up with testtools 2.0.0

This can bee seen:
[1] Install from test-requirements: http://logs.openstack.org/14/246114/10/check/gate-requirements-python27/deec2ab/tox/py27-1.log
[2] Upgrade form requirements: http://logs.openstack.org/14/246114/10/check/gate-requirements-python27/deec2ab/tox/py27-2.log

Then we get test failures as seen http://logs.openstack.org/14/246114/10/check/gate-requirements-python27/deec2ab/console.html

There are a couple of fixes to this
1) Across the OpenStack code base manually and somewhat artificially for testools<2.0.0
2) Add a cap in the requirements for fixtures 1.2.0 of "testtools>=0.9.22,<2.0.0" and release that as 1.2.1

I believe that option 2 is the correct solution as fixtures 1.2.0 is NOT compatible with testtoools 2.0.0

[Robert Collins (lifeless)]

So, the issues are that
a) pip doesn't do recursive resolving, so once you pip install fixtures, the dep from testtools to fixtures is ignored as fixtures is already selected. https://github.com/pypa/pip/issues/988
b) pypi is immutable, we can't alter the definition of fixtures 1.2.0, so the dependency it has, it always will have. We could issue a 1.2.1 with a dependency of testtools >= 2.0.0 which would address the issue for users of fixtures in general

fixtures 1.2.0 *is* compatible with testtools 2.0.0 - the things in the testtools API that it depends on are present in 2.0.0. The compat issue you're running into is on the other side - so capping it would be odd at best.

c) I recommend not using 'pip install -U fixtures==1.2.0' - its a command that tells pip to install the latest version of every dependency, rather than trusting things will have appropriate minimums. If you want testtools 1.9.0 and fixtures 1.2.0, install those things and don't ask pip to grab the latest versions of them - which -U does.

[Tony Breeds (o-tony)]

So Just to be certain we're on the same page.

In the openstack/requirements repo we're trying to land: https://review.openstack.org/#/c/246114/10

We correctly setup the venv with fixtures 1.2.0 and testtools 1.9.0 and then we do
pip install -U -e . /the/path/to/requirements/review

And it's the final step that then upgrades testtools (because of the -U) So your suggested fix is to modify the tox.ini config to remove the -U?

Also with at same review as the code in question
virtualenv foo
foo/bin/pip install -e .

WIll install fixtures 1.2.0 and testtools 2.0.0 and when you have these 2 installed pkg_resources detects a conflict as testtools requires fixtures >=1.3.0

So as I said we could do option 1 but requirements doesn't really require testtools

[Robert Collins (lifeless)]

openstack/requirements uses testtools for its testing; I see no harm in listing testtools directly in test-requirements; we're hoping to get pip's bug fixed this quarter :)

The removal of -U probably needs wider discussion than this bug - but it is my first recommendation.

[Tony Breeds (o-tony)]

we're talking about adding " testtools>=0.9.36,!=1.2.0,<2.0.0" to requirements.txt in openstack/requirements and *also* adding that line to test-requirements.txt in 34 other projects in the openstack/* namespace. This issue affects more than just openstack/requirements. It affects *all* fixtures users that, for whatever reason, must use fixtures <1.3.0.

I understand, at a superficial level, that this is fallout from pip lacking a fully recursive depsolver,

In Comment 2 item b you say "We could issue a 1.2.1 dependency of testtools >= 2.0.0 which would address the issue for users of fixtures in general" Modulo s/>=/</, that's what I'm requesting. Is it 100% correct? probably not. Is it a reasonable compromise given the tools are what they are today? IMO yes

[Tony Breeds (o-tony)]

I've updated https://review.openstack.org/#/c/246114/ to include testtools in requirements.txt as I suspect that my claim about needing the edit 34 other projects is incorrect.

[Robert Collins (lifeless)]

Why can't you use fixtures 1.3 ?

[Tony Breeds (o-tony)]

From: dhellmann via https://review.openstack.org/#/c/201354/
---
Cap fixtures<1.3.0

More recent versions of fixtures introduce behavioral changes that,
while not strictly API breaks, do break the tests for some projects where
fixtures are cleaned up more than once. We could backport the fixes we
have made in master, but that would introduce an excessive amount of
churn in the stable branch.
---

[Matt Riedemann (mriedem)]

I haven't read all of the discussion here, but in reading the bug description I came to the same conclusion as option 2 above:

2) Add a cap in the requirements for fixtures 1.2.0 of "testtools>=0.9.22,<2.0.0" and release that as 1.2.1

But yeah it's really just a problem for openstack on stable/kilo because we capped fixtures<1.3.0:

https://github.com/openstack/requirements/blob/2e4efd8f9bd451249875e47bc641d58da31f53c6/global-requirements.txt#L217

But given testtools>=2.0.0 requires fixtures>=1.3.0, it doesn't seem terrible to cap testtools<2.0.0 in a fixtures 1.2.1 release as suggested here.

[Ihar Hrachyshka (ihar-hrachyshka)]

Any progress on that one? Neutron kilo gate is completely broken due to the issue.

[J N (jn826t)]

As a temporary workaround, I had to also change the METADATA for testtools-2.0.0.dist-info

[Matt Riedemann (mriedem)]

1.2.1 is out: https://pypi.python.org/pypi/fixtures/1.2.1

[Matt Riedemann (mriedem)]

https://github.com/testing-cabal/fixtures/commit/255f9cd00c9bc4dfd90cf9073e6c935d3009e163