pip functionality has been changed causing deployment being failed

[Maaz]

Earlier We had done a POC for Openstack deployment on using project Openstack-ansible HEAD -> stable/victoria, tag: 22.1.1, origin/stable/victoria

Everything went fine and we were able to successfully deploy the openstack services.
We cloned the repository with latest stable/Victoria code commit f1313b5511a7310c1f6435afe925a747c1352d55 (HEAD -> stable/victoria, tag: 22.3.3, origin/stable/victoria)

and when we tried to redeploy again it for production use, we see it is failing to deploy first service keystone, with pip error.

Looks like pip functionality is changes and stable code no longer works as expected.
Below is the error and it contains what is getting failed discussion @ https://github.com/pypa/pip/issues/8210

I was able to track this issue in devstack project where they has applied some fix.
Just for ref : https://review.opendev.org/c/openstack/devstack/+/802642

==========================================================================================
fatal: [nimbus-js-test-1]: FAILED! => {"attempts": 5, "changed": false, "cmd": ["/openstack/venvs/keystone-22.1.1/bin/pip3", "install", "-U", "--constraint", "/openstack/venvs/keystone-22.1.1/global-constraints.txt", "--constraint", "/openstack/venvs/keystone-22.1.1/constraints.txt", "--pre", "--log", "/var/log/python_venv_build.log", "keystone", "ldappool", "osprofiler", "pymemcache", "PyMySQL", "pyngus", "python-memcached", "python-openstackclient", "systemd-python", "uWSGI"], "msg": "stdout: Ignoring immutables: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring importlib-metadata: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring contextvars: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring dataclasses: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring importlib-resources: markers 'python_version == \"3.6\"' don't match your environment\nIgnoring futures: markers 'python_version == \"2.7\"' don't match your environment\nCollecting keystone\n Cloning https://opendev.org/openstack/keystone (to revision 5b860e0b3b4e318b91325996156bae3f99abd6c7) to ./pip-install-8d4f8vdv/keystone\n\n:stderr: DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210.\nERROR: Command errored out with exit status 128: git clone -q https://opendev.org/openstack/keystone /tmp/pip-install-8d4f8vdv/keystone Check the logs for full command output.\nWARNING: You are using pip version 20.2.3; however, version 21.3.1 is available.\nYou should consider upgrading via the '/openstack/venvs/keystone-22.1.1/bin/python -m pip install --upgrade pip' command.\n"}

===========================================================================================

[Dmitriy Rabotyagov (noonedeadpunk)]

Hi!

I believe this has nothing to do with mentioned pip change since we use pip version prior to the one that has change you mention.
Based on the output you provided, real error is:
`ERROR: Command errored out with exit status 128: git clone -q https://opendev.org/openstack/keystone /tmp/pip-install-8d4f8vdv/keystone Check the logs for full command output.`

If we were betting, I'd say that you most likely have outdated ca-certificates package on your hosts, which prevents git to clone the repo. IF that's the case, you can update them with ad-hoc command:

cd /opt/openstack-ansible; ansible -m package -a "name=ca-certificates state=latest update_cache=yes only_upgrade=yes" all

[Jonathan Rosser (jrosser)]

The migration to the new pip resolver has been handled properly in openstack-ansible so you should not see a failure related to that.

The log message relating to the pip resolver changes is a deprecation rather than an error. pip 20.3 enables the new resolver and in the victoria branch we pin the version of pip to 20.2.3 which is before this change.

Your error seems related to cloning the keystone git repo "ERROR: Command errored out with exit status 128: git clone -q https://opendev.org/openstack/keystone /tmp/pip-install-8d4f8vdv/keystone Check the logs for full command output"

Please check the detail in the venv build log on the host which tried to build keystone, I think you will find better clues there.

[Maaz (maaz0123)]

Hi Dmitriy,

Thanks for valuable input, only that outdated certificate was causing that issue, after certificate update it got fixed.