Allow venvs to be built with disparate python requirements

[Jesse Pretorius]

Rally does not have its requirements managed by the OpenStack global requirements process. Both it and tempest are likely to have requirements outside of the bounds of an OpenStack service's stable branch requirements.

In Ocata there is a higher likelihood that more and more services will have disparate requirements.

For this situation, the repo_build process completely breaks when trying to build a venv for the service because the process is forced to use the locally built requirements only... and we force the requirements down to a single set. In other words - we don't allow the repo to contain more than one wheel by the same name.

We need to change the process to allow more than one wheel of the same name to be built so that we can cater for disparate requirements.

[Jean-Philippe Evrard (jean-philippe-evrard)]

The emergency state is now reduced, due to upstream fixes. This should still be considered as a priority though.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/386759

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/386765

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/391029
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-plugins/commit/?id=7abcaf91972fe3d0e9af98f36b5ed479d1b696cc
Submitter: Jenkins
Branch: master

commit 7abcaf91972fe3d0e9af98f36b5ed479d1b696cc
Author: Kevin Carter <email address hidden>
Date: Thu Oct 27 00:37:10 2016 -0500

    Add the ability to ignore requirements from a given package

    Some packages simply do not play well with others. One such case is
    "tempest" where time and time again it's unbound requirements break
    everything. This change allows the package indexer to examine remote
    repos and exclude packages from the global package building list by
    using the metadata value ``ignorerequirements=True``. When a package is
    ignored it can still be built within in a venv or used by referencing
    the git repos however it will no longer be part of the global wheel
    building package list.

    Change-Id: Iacf15eea14a2cd9c98bbb59d99e2a1ea94bd0a5b
    Closes-Bug: #1631992
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/386765
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=baab4e2e7c5d96a3cd52bcf2703b12f064f63b73
Submitter: Jenkins
Branch: master

commit baab4e2e7c5d96a3cd52bcf2703b12f064f63b73
Author: Kevin Carter <email address hidden>
Date: Fri Oct 14 15:23:07 2016 -0500

    Updates to support ignored packages and external indexes

    This change ensures that a venv create process can work when a
    dependency is crafted outside of the OpenStack global requirements
    system. The change keys off of the original package url setting the
    isolated perameter should "withindex=True" be found.

    The requirements.txt.j2 template file has been updated to allow
    packages to be ignored from wheel building should the
    `ignorerequirement=True` property be set within a given remote package.
    This ability is important as some packages, generally external testing
    repos, do not use requirement boundries and are known to break
    given enough time.

    Tagged tasks that MUST always be run have had tags set to "always".

    Change-Id: I5d11aca84dcb74c77f3d0e3f31ce6546458a3e84
    Closes-Bug: #1631992
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/386759
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=f28ddbbe3134a7a4d3d4a71a0721428f86ad2e03
Submitter: Jenkins
Branch: master

commit f28ddbbe3134a7a4d3d4a71a0721428f86ad2e03
Author: Kevin Carter <email address hidden>
Date: Fri Oct 14 14:51:30 2016 -0500

    Updates to support ignored packages and external indexes

    Some OpenStack services, generally testing repos, do not strictly
    follow the requirements g-r sets. This change allows these services to
    build using the `withindex=True` and `ignorerequirements=True`
    properties. These properties will allow the package to fetch
    requirements as needed without potentially breaking a deployment due to
    unbound dependencies.

    Depends-On: I5d11aca84dcb74c77f3d0e3f31ce6546458a3e84
    Change-Id: I8651702656aa6d2861b45dc4b111187cfbac4719
    Partial-Bug: #1631992
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-plugins 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/446718

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/446719

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/446719
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=a9b1c49be56eb93167c71a5d764fbcc17cc2a976
Submitter: Jenkins
Branch: stable/newton

commit a9b1c49be56eb93167c71a5d764fbcc17cc2a976
Author: Kevin Carter <email address hidden>
Date: Fri Oct 14 15:23:07 2016 -0500

    Updates to support ignored packages and external indexes

    This change ensures that a venv create process can work when a
    dependency is crafted outside of the OpenStack global requirements
    system. The change keys off of the original package url setting the
    isolated perameter should "withindex=True" be found.

    The requirements.txt.j2 template file has been updated to allow
    packages to be ignored from wheel building should the
    `ignorerequirement=True` property be set within a given remote package.
    This ability is important as some packages, generally external testing
    repos, do not use requirement boundries and are known to break
    given enough time.

    Tagged tasks that MUST always be run have had tags set to "always".

    Change-Id: I5d11aca84dcb74c77f3d0e3f31ce6546458a3e84
    Closes-Bug: #1631992
    Signed-off-by: Kevin Carter <email address hidden>
    (cherry picked from commit baab4e2e7c5d96a3cd52bcf2703b12f064f63b73)

[OpenStack Infra (hudson-openstack)]

Change abandoned by Kevin Carter (cloudnull) (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/446718
Reason: abandoned as this was not needed.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 14.2.0 release.