[CI][CS8][Train] diskimage builder requires a different Python: 3.6.8 not in '>=3.8

[Dariusz Smigiel]

periodic-tripleo-centos-8-buildimage-overcloud-full-train job started failing somewhere between:
2022-06-06 11:10:41 and 2022-06-07 11:07:49 [1] with an issue to install requirements.

2022-06-07 20:42:05.420896 | primary | ERROR
2022-06-07 20:42:05.421161 | primary | {
2022-06-07 20:42:05.421210 | primary | "msg": "stdout: Looking in indexes: http://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/pypi/simple, http://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/wheel/centos-8-x86_64\nCollecting git+file:/home/zuul/src/opendev.org/openstack/diskimage-builder\n Cloning file:///home/zuul/src/opendev.org/openstack/diskimage-builder to ./pip-req-build-5l2z1_3p\n Resolved file:///home/zuul/src/opendev.org/openstack/diskimage-builder to commit fe0e5324d4248d114660ec35111ae9601e05b95b\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'done'\nProcessing /home/zuul/src/opendev.org/openstack/tripleo-ansible\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'done'\nProcessing /home/zuul/src/opendev.org/openstack/tripleo-common\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'done'\nProcessing /home/zuul/src/opendev.org/openstack/python-tripleoclient\n Preparing metadata (setup.py): started\n Preparing metadata (setup.py): finished with status 'done'\nRequirement already satisfied: networkx>=2.3.0 in /home/zuul/workspace/venv/lib/python3.6/site-packages (from diskimage-builder==3.21.2.dev11) (2.3)\nRequirement already satisfied: pbr!=2.1.0,>=2.0.0 in /home/zuul/workspace/venv/lib/python3.6/site-packages (from diskimage-builder==3.21.2.dev11) (5.4.3)\nRequirement already satisfied: PyYAML>=3.12 in /home/zuul/workspace/venv/lib/python3.6/site-packages (from diskimage-builder==3.21.2.dev11) (5.1.2)\nRequirement already satisfied: stevedore>=1.20.0 in /home/zuul/workspace/venv/lib/python3.6/site-packages (from diskimage-builder==3.21.2.dev11) (1.31.0)\nRequirement already satisfied: flake8<4.0.0,>=3.6.0 in /home/zuul/workspace/venv/lib/python3.6/site-packages (from diskimage-builder==3.21.2.dev11) (3.9.2)\n\n:stderr: Running command git clone --filter=blob:none -q file:///home/zuul/src/opendev.org/openstack/diskimage-builder /tmp/pip-req-build-5l2z1_3p\n warning: filtering not recognized by server, ignoring\nERROR: Package 'diskimage-builder' requires a different Python: 3.6.8 not in '>=3.8'\n"

[1]: https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-8-buildimage-overcloud-full-train

[2]: https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-full-train/51020bb/job-output.txt
[3]: https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-full-train/9f0bf78/job-output.txt
[4]: https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-full-train/cc16561/job-output.txt
[5]: https://logserver.rdoproject.org/a7/a70a36db14e73008f5cfef17178170f2fbbc9b33/openstack-periodic-integration-stable4/periodic-tripleo-centos-8-buildimage-overcloud-full-train/cf26e13/job-output.txt
[6]: https://logserver.rdoproject.org/a7/a70a36db14e73008f5cfef17178170f2fbbc9b33/openstack-periodic-integration-stable4/periodic-tripleo-centos-8-buildimage-overcloud-full-train/64ebaa3/job-output.txt

[Ronelle Landy (rlandy)]

Possibly related change: https://github.com/openstack/diskimage-builder/commit/fe0e5324d4248d114660ec35111ae9601e05b95b

[Ronelle Landy (rlandy)]

https://github.com/openstack/python-tripleoclient/blob/stable/train/setup.cfg

    Programming Language :: Python
    Programming Language :: Python :: 2
    Programming Language :: Python :: 2.7
    Programming Language :: Python :: 3
    Programming Language :: Python :: 3.6
    Programming Language :: Python :: 3.7

[Dariusz Smigiel (smigiel-dariusz)]

dib is dropping support for <py3.8 [1] with Zed release.
dib is branchless, so we're pulling master.

tripleoclient [2] for train still supports py3.6 and py3.7

[1]: https://github.com/openstack/diskimage-builder/commit/fe0e5324d4248d114660ec35111ae9601e05b95b
[2]: https://github.com/openstack/python-tripleoclient/blob/stable/train/setup.cfg#L22

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845081

[Ian Wienand (iwienand)]

We did drop this, some comments in [1]. I was a little concerned that we'd hit something like this, we did hold off on merging similar some time ago, but with the Zed release it seemed more like something we could take on.

Due to our branchless nature, we don't really have a way to keep this going separately. However I can understand that running from master dib is also important because sometimes upstream things change that we fix -- so often fixes aren't really new features in dib, but just keeping what is going working.

I don't know what the answer is ... we *can* keep the 3.6 jobs and revert the removal (and keep the 3.10) -- it's not like we use a ton of python features. But also it's an obsolete python ... I haven't released dib yet, and won't until we come to some sort resolution.

Can train pin to the prior dib release?

[1] https://review.opendev.org/c/openstack/diskimage-builder/+/836229

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845081
Reason: Not working as expected

[Steve Baker (steve-stevebaker)]

We could pin train to an older dib, but we need to consider the likelihood that train will need a dib fix in the future which it can't get without a stable branch.

How about the periodic-tripleo-centos-8-buildimage-overcloud-full-train job install and use python3.8? One way would be a task which detects if default python is < 3.8, installs 3.8, then switches the default to it using "alternatives"

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845137

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845081

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845229

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845229
Committed: https://opendev.org/openstack/tripleo-quickstart/commit/aa8efa86a561b5544d8221bd2e657f2de53ae415
Submitter: "Zuul (22348)"
Branch: master

commit aa8efa86a561b5544d8221bd2e657f2de53ae415
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Thu Jun 9 12:58:39 2022 +0530

    [Workaround] Pull openstack-selinux from common component repo

    Current we need latest openstack-selinux to work with
    libselinux-4.3-1.
    Latest openstack-selinux is available in tripleo-ci-testing
    and we need a promotion to unblock the gate.

    In order to temprorily unblock the upstream gate, we are
    pulling latest openstack-selinux from common component
    repo.

    Related-Bug: #1977888

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: Ibce6ce594c0461f646bd82c5eb3d002000381756

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845274

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ci/+/845081
Committed: https://opendev.org/openstack/tripleo-ci/commit/6d7eaa96f73f348ecdd8b63c8a31ad64e2411119
Submitter: "Zuul (22348)"
Branch: master

commit 6d7eaa96f73f348ecdd8b63c8a31ad64e2411119
Author: Chandan Kumar (raukadah) <email address hidden>
Date: Wed Jun 8 09:36:15 2022 +0530

    Install Python38 on CentOS-8

    DIB is a branchless project and it gets pulled from master.
    https://review.opendev.org/c/openstack/diskimage-builder/+/836229
    drops the py36 support and is breaking cs8 image build.

    Installing Python38[1] will fix the issue.

    It also drops python3-tripleoclient and python3-devel installation
    from main.yml. It is already taken care by release file.

    py38 does not works well with Ansible on CentOS-8[2].

    So we also set ensure_pip_virtualenv_command var to pip ansible module
    to explicitly use python_version based on need.

    [1]. https://pkgs.org/download/python38
    [2]. https://github.com/ansible/ansible/issues/67083

    Closes-Bug: #1977888

    Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/845189

    Signed-off-by: Chandan Kumar (raukadah) <email address hidden>
    Change-Id: Ib87eb7bbe6b523e3bf78c7f870cff96d5b5dd7e1

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845352

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ronelle Landy <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845352
Reason: will revert full patch

[Ronelle Landy (rlandy)]

Waiting on proper fix to merge: https://review.opendev.org/c/openstack/diskimage-builder/+/845217

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-quickstart/+/845137
Reason: no longer needed

[OpenStack Infra (hudson-openstack)]

Change abandoned by "chandan kumar <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845544

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/846113

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-ci/+/846113
Committed: https://opendev.org/openstack/tripleo-ci/commit/c5254c2224a80ebfbd33b21eb1aaa7686d91dc97
Submitter: "Zuul (22348)"
Branch: master

commit c5254c2224a80ebfbd33b21eb1aaa7686d91dc97
Author: chandan kumar <email address hidden>
Date: Thu Jun 16 05:53:31 2022 +0000

    Revert "Install Python38 on CentOS-8"

    This reverts commit 6d7eaa96f73f348ecdd8b63c8a31ad64e2411119.

    Closes-Bug: #1977888

    Reason for revert: https://review.opendev.org/c/openstack/diskimage-builder/+/845217

    Change-Id: I55a9feb105e13d6ab99810f09459bd9760cf1e9e

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Ghanshyam <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/tripleo-ci/+/845274
Reason: TrieplO project is retiring now, for details, please see https://review.opendev.org/c/openstack/governance/+/905145 or reach out to OpenStack TC.