ImportError in pip wrappers after upgrade

[Moises Emilio Benzan Mora]

The K8s-suite pins down a pip version to use in the virtualenv for tox tests as [22.0.4](https://github.com/charmed-kubernetes/jenkins/blob/1.26/tox.ini#L9), however then the tox test environment configuration commands trigger an indirect pip upgrade when it calls [`pip-sync`](https://github.com/charmed-kubernetes/jenkins/blob/1.26/tox.ini#L13). This gets pip into a known error state as shown on [pypa/pip#5599](https://github.com/pypa/pip/issues/5599).

The proposed solution is to either:
a) unpin the initial tox dependency on pip==22.0.4, or
b) properly pin down pip on the `requirements.txt` so that the subsequent call to `pip-sync` won't attempt to auto-upgrade.

Error seen in the following runs:
1) https://solutions.qa.canonical.com/v2/testruns/f9707675-48ea-492f-bff4-74b9bea066dd/
Logs: https://oil-jenkins.canonical.com/artifacts/f9707675-48ea-492f-bff4-74b9bea066dd/index.html
Jenkins logs: https://oil-jenkins.canonical.com/job/fce_build/8074//consoleFull

2) https://solutions.qa.canonical.com/v2/testruns/651de4a2-5580-4aa9-a656-383e19719515
Logs: https://oil-jenkins.canonical.com/artifacts/651de4a2-5580-4aa9-a656-383e19719515/index.html
Jenkins logs: https://oil-jenkins.canonical.com/job/fce_build/8065//consoleFull

[Alexander Balderson (asbalderson)]

adding the output from pip

2022-12-02-05:55:11 root ERROR [localhost] STDOUT follows:
py3 create: /home/ubuntu/k8s-validation/.tox/py3
py3 installdeps: pip == 22.0.4, pip-tools
py3 installed: build==0.9.0,click==8.1.3,packaging==21.3,pep517==0.13.0,pip-tools==6.11.0,pkg_resources==0.0.0,pyparsing==3.0.9,tomli==2.0.1
py3 run-test-pre: PYTHONHASHSEED='3359271779'
py3 run-test: commands[0] | pip install cffi flake8
Collecting cffi
  Downloading cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 442.7/442.7 KB 1.1 MB/s eta 0:00:00
Collecting flake8
  Downloading flake8-6.0.0-py2.py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.8/57.8 KB 592.0 kB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 1.3 MB/s eta 0:00:00
Collecting mccabe<0.8.0,>=0.7.0
  Downloading mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
Collecting pycodestyle<2.11.0,>=2.10.0
  Downloading pycodestyle-2.10.0-py2.py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.3/41.3 KB 16.4 MB/s eta 0:00:00
Collecting pyflakes<3.1.0,>=3.0.0
  Downloading pyflakes-3.0.1-py2.py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 7.8 MB/s eta 0:00:00
Installing collected packages: pyflakes, pycparser, pycodestyle, mccabe, flake8, cffi
Successfully installed cffi-1.15.1 flake8-6.0.0 mccabe-0.7.0 pycodestyle-2.10.0 pycparser-2.21 pyflakes-3.0.1
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/home/ubuntu/k8s-validation/.tox/py3/bin/python -m pip install --upgrade pip' command.
py3 run-test: commands[1] | pip-sync /home/ubuntu/k8s-validation/requirements.txt
Traceback (most recent call last):
  File "/home/ubuntu/k8s-validation/.tox/py3/bin/pip-sync", line 5, in <module>
    from piptools.scripts.sync import cli
  File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/piptools/scripts/sync.py", line 16, in <module>
    from .. import sync
  File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/piptools/sync.py", line 15, in <module>
    from ._compat.pip_compat import Distribution, dist_requires
  File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/piptools/_compat/__init__.py", line 3, in <module>
    from .pip_compat import PIP_VERSION, parse_requirements
  File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/piptools/_compat/pip_compat.py", line 52, in <module>
    uses_pkg_resources = _uses_pkg_resources()
  File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/piptools/_compat/pip_compat.py", line 46, in _uses_pkg_resources
    from pip._internal.metadata import select_backend
ImportError: cannot import name 'select_backend' from 'pip._internal.metadata' (/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/pip/_internal/metadata/__init__.py)
ERROR: InvocationError for command /home/ubuntu/k8s-validation/.tox/py3/bin/pip-sync requirements.txt (exited with code 1)

[Adam Dyess (addyess)]

Yes, i've experienced the same situation. I'll take a look at it

08:25:38 py38 create: /var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38
08:25:38 py38 installdeps: pip == 22.0.4, pip-tools
08:25:42 py38 installed: build==0.9.0,click==8.1.3,packaging==21.3,pep517==0.13.0,pip-tools==6.11.0,pkg_resources==0.0.0,pyparsing==3.0.9,tomli==2.0.1
08:25:42 py38 run-test-pre: PYTHONHASHSEED='4262616393'
08:25:42 py38 run-test: commands[0] | pip install cffi flake8
08:25:42 Collecting cffi
08:25:42 Using cached cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
08:25:42 Collecting flake8
08:25:42 Using cached flake8-6.0.0-py2.py3-none-any.whl (57 kB)
08:25:42 Collecting pycparser
08:25:43 Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
08:25:43 Collecting pycodestyle<2.11.0,>=2.10.0
08:25:43 Using cached pycodestyle-2.10.0-py2.py3-none-any.whl (41 kB)
08:25:43 Collecting mccabe<0.8.0,>=0.7.0
08:25:43 Using cached mccabe-0.7.0-py2.py3-none-any.whl (7.3 kB)
08:25:43 Collecting pyflakes<3.1.0,>=3.0.0
08:25:43 Using cached pyflakes-3.0.1-py2.py3-none-any.whl (62 kB)
08:25:43 Installing collected packages: pyflakes, pycparser, pycodestyle, mccabe, flake8, cffi
08:25:43 Successfully installed cffi-1.15.1 flake8-6.0.0 mccabe-0.7.0 pycodestyle-2.10.0 pycparser-2.21 pyflakes-3.0.1
08:25:43 WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
08:25:43 You should consider upgrading via the '/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/bin/python -m pip install --upgrade pip' command.
08:25:43 py38 run-test: commands[1] | pip-sync /var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/requirements.txt
08:25:43 Traceback (most recent call last):
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/bin/pip-sync", line 5, in <module>
08:25:43 from piptools.scripts.sync import cli
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/lib/python3.8/site-packages/piptools/scripts/sync.py", line 16, in <module>
08:25:43 from .. import sync
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/lib/python3.8/site-packages/piptools/sync.py", line 15, in <module>
08:25:43 from ._compat.pip_compat import Distribution, dist_requires
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/lib/python3.8/site-packages/piptools/_compat/__init__.py", line 3, in <module>
08:25:43 from .pip_compat import PIP_VERSION, parse_requirements
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/lib/python3.8/site-packages/piptools/_compat/pip_compat.py", line 52, in <module>
08:25:43 uses_pkg_resources = _uses_pkg_resources()
08:25:43 File "/var/lib/jenkins/slaves/jenkins-slave-focal-ps5-8/workspace/promote-charms/.tox/py38/lib/python3.8/site-packages/piptools/_compat/pip_compat.py", line 46, in _uses_pkg_resources
08:25:43 from pip._internal.metadata import se...

[Adam Dyess (addyess)]

Proposed: https://github.com/charmed-kubernetes/jenkins/pull/1123

[Adam Dyess (addyess)]

1.26 tags has been updated