Python wheel build is skipped on metal

[James Denton]

Distro: OpenStack-Ansible
Version: Zed (Yoga)
Point: Master (25.1.0)

In a multi-node *metal* deployment, the utility-install.yml playbook fails with:

TASK [python_venv_build : Slurp up the constraints file for later re-deployment] ***********************************************
fatal: [infra2 -> infra1(10.0.236.100)]: FAILED! => {"changed": false, "msg": "file not found: /var/www/repo/os-releases/25.1.0.
dev68/ubuntu-20.04-x86_64/requirements/utility-25.1.0.dev68-constraints.txt"}
fatal: [infra3 -> infra1(10.0.236.100)]: FAILED! => {"changed": false, "msg": "file not found: /var/www/repo/os-releases/25.1.0.dev68/ubuntu-20.04-x86_64/requirements/utility-25.1.0.dev68-constraints.txt"}

Full log: https://paste.opendev.org/show/bTs7A7RPieBBbmYDYGYp/

This appears to be the result of venv_wheel_build_enable=false for infra1 (the deploy host) and venv_wheel_build_enable=true for infra2/3, and an attempt to slurp files that weren't created.

That's a result of this logic:

https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/defaults/main.yml#L119

venv_wheel_build_enable: "{{ venv_build_host != inventory_hostname }}"

Changing != to == allows this to work as expected, but is probably against the spirit of the original change.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/857897

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/858258

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/858258
Committed: https://opendev.org/openstack/openstack-ansible-os_placement/commit/6084c248fcae02c413133329b705678cd75c1bfe
Submitter: "Zuul (22348)"
Branch: master

commit 6084c248fcae02c413133329b705678cd75c1bfe
Author: Dmitriy Rabotyagov <email address hidden>
Date: Mon Sep 19 09:54:35 2022 +0200

    Install git into placement containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859162

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859163

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859164

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/857897
Committed: https://opendev.org/openstack/ansible-role-python_venv_build/commit/4a04d08eb4f10162a92c84f1fc543e7b110f53f8
Submitter: "Zuul (22348)"
Branch: master

commit 4a04d08eb4f10162a92c84f1fc543e7b110f53f8
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu Sep 15 17:44:05 2022 +0200

    Change default value for venv_wheel_build_enable

    With fixing multi-distro support logic, venv_wheel_build_enable logic
    stopped working for metal deployments, that have more then 1 host.
    Currently, we were checking, that we are not building wheels if build_host
    is the same as inventory_hostname. Which is now true for all metal
    deployments. However, other hosts do expect wheels to present, which
    results in installation failure.

    According to the original idea we need to build wheels only when we have
    more then 1 host with same arch/distro combination otherwise it does
    not make sense to build wheels and faster to install tarball.

    To achieve that we build mapping of all distro/arch combinations and
    enable wheels building only when there distro-arch combination
    is not unique for the host we're playing against.

    Despite it might be overkill, as scenario of having only single host
    with uniqe distro-arch combination, and overhead of building wheels for
    it would be quite small, alternative would be to hardcode `True` for
    venv_wheel_build_enable.

    Closes-Bug: #1989506
    Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/858258
    Change-Id: I762e36acf76729fd61f28ca1b03bc9f562b5db0a

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/859231

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859162
Committed: https://opendev.org/openstack/openstack-ansible-os_placement/commit/bc9773f0b62c95b0654d3e41d424a8e380107b96
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit bc9773f0b62c95b0654d3e41d424a8e380107b96
Author: Dmitriy Rabotyagov <email address hidden>
Date: Mon Sep 19 09:54:35 2022 +0200

    Install git into placement containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    (cherry picked from commit 6084c248fcae02c413133329b705678cd75c1bfe)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859163
Committed: https://opendev.org/openstack/openstack-ansible-os_placement/commit/e2254be8827d2a9dca651c97e5eff1160a48277a
Submitter: "Zuul (22348)"
Branch: stable/xena

commit e2254be8827d2a9dca651c97e5eff1160a48277a
Author: Dmitriy Rabotyagov <email address hidden>
Date: Mon Sep 19 09:54:35 2022 +0200

    Install git into placement containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    (cherry picked from commit 6084c248fcae02c413133329b705678cd75c1bfe)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859164
Committed: https://opendev.org/openstack/openstack-ansible-os_placement/commit/f8b256545bdf58cbc5657ce8dab2e82d3a52afe1
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit f8b256545bdf58cbc5657ce8dab2e82d3a52afe1
Author: Dmitriy Rabotyagov <email address hidden>
Date: Mon Sep 19 09:54:35 2022 +0200

    Install git into placement containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    (cherry picked from commit 6084c248fcae02c413133329b705678cd75c1bfe)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/859231
Committed: https://opendev.org/openstack/ansible-role-python_venv_build/commit/8a9ababb80666546c42d0655e2e728858ae50d00
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 8a9ababb80666546c42d0655e2e728858ae50d00
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu Sep 15 17:44:05 2022 +0200

    Change default value for venv_wheel_build_enable

    With fixing multi-distro support logic, venv_wheel_build_enable logic
    stopped working for metal deployments, that have more then 1 host.
    Currently, we were checking, that we are not building wheels if build_host
    is the same as inventory_hostname. Which is now true for all metal
    deployments. However, other hosts do expect wheels to present, which
    results in installation failure.

    According to the original idea we need to build wheels only when we have
    more then 1 host with same arch/distro combination otherwise it does
    not make sense to build wheels and faster to install tarball.

    To achieve that we build mapping of all distro/arch combinations and
    enable wheels building only when there distro-arch combination
    is not unique for the host we're playing against.

    Despite it might be overkill, as scenario of having only single host
    with uniqe distro-arch combination, and overhead of building wheels for
    it would be quite small, alternative would be to hardcode `True` for
    venv_wheel_build_enable.

    Closes-Bug: #1989506
    Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/859162
    Change-Id: I762e36acf76729fd61f28ca1b03bc9f562b5db0a
    (cherry picked from commit 4a04d08eb4f10162a92c84f1fc543e7b110f53f8)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/864526

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/864526
Committed: https://opendev.org/openstack/openstack-ansible-os_heat/commit/74cf65fd75f3489282803c9164f2442ba44125e4
Submitter: "Zuul (22348)"
Branch: master

commit 74cf65fd75f3489282803c9164f2442ba44125e4
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Nov 15 13:57:42 2022 +0100

    Install git into placement containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    Closes-Bug: #1995536

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/865564

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/865565

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/865564
Committed: https://opendev.org/openstack/openstack-ansible-os_heat/commit/38ba2096338c9f3de53cf027518ee6ca338611fb
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 38ba2096338c9f3de53cf027518ee6ca338611fb
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Nov 15 13:57:42 2022 +0100

    Install git into heat containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    Closes-Bug: #1995536
    (cherry picked from commit 74cf65fd75f3489282803c9164f2442ba44125e4)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/865565
Committed: https://opendev.org/openstack/openstack-ansible-os_heat/commit/c452633ef312e40f27f95a9858513b7fcd454704
Submitter: "Zuul (22348)"
Branch: stable/xena

commit c452633ef312e40f27f95a9858513b7fcd454704
Author: Dmitriy Rabotyagov <email address hidden>
Date: Tue Nov 15 13:57:42 2022 +0100

    Install git into heat containers

    If venv_wheel_build_enable is defined to False, placement will fail to
    clone and install repositories due to missing git binary.

    Change-Id: If1e3eec0c558d1472da7bc3a4e87825e36ba4fdc
    Related-Bug: #1989506
    Closes-Bug: #1995536
    (cherry picked from commit 74cf65fd75f3489282803c9164f2442ba44125e4)