Newton: repo build still broken with optional components (swift)

[Bjoern]

In the repo build process I still have the issue that a environment without swift, still leads to build errors:

1st the git clone for the swift repo is skipped:

TASK [repo_build : Clone git repositories asynchronously] **********************
...
skipping: [infra01_repo_container-d66881bb] => (item={u'name': u'swift', u'project_group': u'swift_all', u'fragment': None, u'url': u'https://git.openstack.org/openstack/swift', u'version': u'2db841c907e7d6b1f2097adfab22eb32858f994a', u'original': u'git+https://git.openstack.org/openstack/swift@2db841c907e7d6b1f2097adfab22eb32858f994a#egg=swift&gitname=swift&projectgroup=swift_all', u'egg_name': u'swift'})
...

But later it's referenced inside the requirements file

TASK [repo_build : Create OpenStack-Ansible requirement wheels] ****************
...

  Could not find a tag or branch 'e8c6017ef042c959e355be7f46071964b8fea327', assuming commit.
fatal: '/var/www/repo/openstackgit/swift' does not appear to be a git repository
fatal: Could not read from remote repository.

...

[Jesse Pretorius (jesse-pretorius)]

In your last bug report about this the issue was that you had something else which had swift as a requirement. This is why the requirements file had swift in it. Is this still the case?

[Bjoern (bjoern-t)]

Swift is not a mandatory requirement for me, perhaps we should make it dependable on the having swift nodes inside the inventory. Unlike swift, the swift client components are always needed since I don't know if the components like glance is offloaded to a outside swift cluster.
Hope that makes it clear

[Jesse Pretorius (jesse-pretorius)]

So the issue is that the 'swift' package is set to install in a supplemental role, but there is no git source on the repo server to build the package.

I expect that this can be solved and the selective wheel build can still be taken advantage of by setting this is user_variables:

repo_build_git_selective: no

This will lengthen the repo build time, but not as much as turning off the selective wheel build. It provides the appropriate git source to build the wheel from and should work. Please test and confirm/deny.

[Jesse Pretorius (jesse-pretorius)]

I'll work on confirming this, and making the selective git a little smarter.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/445922

[Jesse Pretorius (jesse-pretorius)]

Switching importance. While this is an edge case and can be seen as a 'feature', I think that it is actually exposing a logic bug which is being fixed in the proposed patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445922
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=7331b821b4e2894a62bd22e7e47e87e04b7d7f71
Submitter: Jenkins
Branch: master

commit 7331b821b4e2894a62bd22e7e47e87e04b7d7f71
Author: Jesse Pretorius <email address hidden>
Date: Wed Mar 15 12:39:54 2017 +0000

    Include git clone for packages in requirements

    Currently if there is a package in requirements which
    needs to be built from one of the git clones, it will
    not work unless the related host group is populated.
    This is due to the fact that the selective git clone
    hinges on the host group population.

    This patch adds an extra check for each git clone to
    validate whether the package is in the requirements
    list too. If it is, then it will clone the git source
    regardless of the host group population.

    Change-Id: Id121a1bcd91f688c8fab41de0b74b5b8196f8b44
    Closes-Bug: #1671958

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/445982

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/445983

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445983
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=6ca099ac57ecadab993eac258f6386f81d56e03f
Submitter: Jenkins
Branch: stable/newton

commit 6ca099ac57ecadab993eac258f6386f81d56e03f
Author: Jesse Pretorius <email address hidden>
Date: Wed Mar 15 12:39:54 2017 +0000

    Include git clone for packages in requirements

    Currently if there is a package in requirements which
    needs to be built from one of the git clones, it will
    not work unless the related host group is populated.
    This is due to the fact that the selective git clone
    hinges on the host group population.

    This patch adds an extra check for each git clone to
    validate whether the package is in the requirements
    list too. If it is, then it will clone the git source
    regardless of the host group population.

    Change-Id: Id121a1bcd91f688c8fab41de0b74b5b8196f8b44
    Closes-Bug: #1671958
    (cherry picked from commit 7331b821b4e2894a62bd22e7e47e87e04b7d7f71)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/445982
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-repo_build/commit/?id=7ea396c94d923fe788c196c88ab9471b9ed33887
Submitter: Jenkins
Branch: stable/ocata

commit 7ea396c94d923fe788c196c88ab9471b9ed33887
Author: Jesse Pretorius <email address hidden>
Date: Wed Mar 15 12:39:54 2017 +0000

    Include git clone for packages in requirements

    Currently if there is a package in requirements which
    needs to be built from one of the git clones, it will
    not work unless the related host group is populated.
    This is due to the fact that the selective git clone
    hinges on the host group population.

    This patch adds an extra check for each git clone to
    validate whether the package is in the requirements
    list too. If it is, then it will clone the git source
    regardless of the host group population.

    Change-Id: Id121a1bcd91f688c8fab41de0b74b5b8196f8b44
    Closes-Bug: #1671958
    (cherry picked from commit 7331b821b4e2894a62bd22e7e47e87e04b7d7f71)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 15.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 14.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible-repo_build 16.0.0.0b1 development milestone.