tox tests do not implement global pins

[Matthew Thode]

this hits stable branches harder than it does master because master has the upper-constraints.txt being updated constantly.

upper-constraints.txt does not track setuptools, this causes the pip install ansible command to pull in the latest setuptools. The latest setuptools requires packaging>=16.8. upper-constraints.txt bounds packaging to ==16.7 so setuptools fails to run when it is run again.

We should add the constraint of global-requirement-pins.txt as additional constraints as it contains a constraint on setuptools.

Patch incoming. I'll also bump the version of packaging in upper-constraints in the requirements repo.

As per http://cdn.pasteraw.com/5jsh909u0nfeocmzjavioi78xdkqr59

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/424283

[Matthew Thode (prometheanfire)]

Not sure about how long this link will last, but this shows the error in newton.

http://cdn.pasteraw.com/5jsh909u0nfeocmzjavioi78xdkqr59

[Miguel Alejandro Cantu (miguel-cantu)]

I'm seeing this as well. I'm using the stable/newton branch.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/424416

[Jesse Pretorius (jesse-pretorius)]

The packages pip, setuptools and wheel are all pinned for an integrated build.

For the ansible bootstrap it's pinned here: https://github.com/openstack/openstack-ansible/blob/master/scripts/scripts-library.sh#L23
For the repo build it's pinned here: https://github.com/openstack/openstack-ansible/blob/master/global-requirement-pins.txt
For the pip_install role it's pinned here: https://github.com/openstack/openstack-ansible/blob/master/playbooks/inventory/group_vars/all.yml#L40

It appears from the bug report that the issue is shown in a tox test, not through bootstrap-ansible. The subject of this bug report is therefore incorrect, and the patch suggested is also trying to apply a patch to the wrong thing. What needs to happen here is to have the pin applied to the tox environments.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/424684

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/424283
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=253ea10d266f5ce4a11249ed3c2f1fe3568a8ace
Submitter: Jenkins
Branch: master

commit 253ea10d266f5ce4a11249ed3c2f1fe3568a8ace
Author: Matthew Thode <email address hidden>
Date: Mon Jan 23 12:33:49 2017 -0600

    Apply global-pins as constraints to ensure complete pinning

    The current pinning method for pip/setuptools/wheel works for the
    initial setup of a host or python venv. However, when a python
    package which has one of these in its requirements is installed
    onto the host or into the venv and the '--upgrade' option is given
    to pip, the package will be upgraded to the latest version available
    within the given constraints.

    As OpenStack's requirements management process does not cater for
    the pip, setuptools or wheel packages we need to ensure that all
    python install tasks prior to the repo being built use our global
    pins as a constraint in addition to the OpenStack uppser constraints.

    As pip's --constraints option can only take a file as an argument
    we have to copy the global pins file to the target which will use
    it. This file can be removed from the host once the repo is built.

    Related-Bug: #1658773
    Closes-Bug: #1658948
    Change-Id: Iccbb2e4a126a1cc7a4c94ab41b7ce8ef54d89990

[Matthew Thode (prometheanfire)]

closing this as upstream decided it wasn't good enough and created their own version of it. Why do I even bother?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/424540
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=e3c8c246e6434d8a3e1ebc4fadcf82047c2f84d4
Submitter: Jenkins
Branch: master

commit e3c8c246e6434d8a3e1ebc4fadcf82047c2f84d4
Author: Jesse Pretorius <email address hidden>
Date: Tue Jan 24 09:42:39 2017 +0000

    Ensure that tox uses global pins

    The tox implementation currently does not make
    use of the global pins as requirements or as
    constraints. This may cause an inconsistent
    pip, setuptools and wheel package set being
    installed when executing tests.

    This patch ensures that the pinned versions are
    installed when setting up the tox venv, and
    ensures that the pins are used as constraints
    in any subsequent package installations.

    Closes-Bug: #1658773
    Change-Id: Id4176f39eeefe3323c35e5f05bca6b208b615088

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/424684
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=c640fcf7d972749eac9656641a5a965570d1410c
Submitter: Jenkins
Branch: stable/mitaka

commit c640fcf7d972749eac9656641a5a965570d1410c
Author: Matthew Thode <email address hidden>
Date: Mon Jan 23 12:33:49 2017 -0600

    Apply global-pins as constraints to ensure complete pinning

    The current pinning method for pip/setuptools/wheel works for the
    initial setup of a host or python venv. However, when a python
    package which has one of these in its requirements is installed
    onto the host or into the venv and the '--upgrade' option is given
    to pip, the package will be upgraded to the latest version available
    within the given constraints.

    As OpenStack's requirements management process does not cater for
    the pip, setuptools or wheel packages we need to ensure that all
    python install tasks prior to the repo being built use our global
    pins as a constraint in addition to the OpenStack uppser constraints.

    As pip's --constraints option can only take a file as an argument
    we have to copy the global pins file to the target which will use
    it. This file can be removed from the host once the repo is built.

    Related-Bug: #1658773
    Closes-Bug: #1658948
    Change-Id: Iccbb2e4a126a1cc7a4c94ab41b7ce8ef54d89990
    (cherry picked from commit 253ea10d266f5ce4a11249ed3c2f1fe3568a8ace)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/424416
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=3859663a6bb4b15705185b6b233e4b66e5728a89
Submitter: Jenkins
Branch: stable/newton

commit 3859663a6bb4b15705185b6b233e4b66e5728a89
Author: Matthew Thode <email address hidden>
Date: Mon Jan 23 12:33:49 2017 -0600

    Apply global-pins as constraints to ensure complete pinning

    The current pinning method for pip/setuptools/wheel works for the
    initial setup of a host or python venv. However, when a python
    package which has one of these in its requirements is installed
    onto the host or into the venv and the '--upgrade' option is given
    to pip, the package will be upgraded to the latest version available
    within the given constraints.

    As OpenStack's requirements management process does not cater for
    the pip, setuptools or wheel packages we need to ensure that all
    python install tasks prior to the repo being built use our global
    pins as a constraint in addition to the OpenStack uppser constraints.

    As pip's --constraints option can only take a file as an argument
    we have to copy the global pins file to the target which will use
    it. This file can be removed from the host once the repo is built.

    Combined backport of:
    - https://review.openstack.org/424283
    - https://review.openstack.org/424540

    Closes-Bug: #1658773
    Closes-Bug: #1658948
    Change-Id: Iccbb2e4a126a1cc7a4c94ab41b7ce8ef54d89990
    (cherry picked from commit 253ea10d266f5ce4a11249ed3c2f1fe3568a8ace)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 15.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 14.0.7 release.