Comment 4 for bug 1908274

Revision history for this message
Marios Andreou (marios-b) wrote :

This is all to do with the new pip resolver [1]. After updating the various locked requirements [2] it get further but eventually times out at [5].

Looking through [5] the entire half hour run is taken by dependency discovery and resolution. For example this [6] is what it does to discover available packages for a single dependency dlrnapi_client (extracted from [5] to illustrate the issue).

There is a relevant warning printed by pip in [5]

 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking

I just bumped to 60 mins to see if it will even complete in that time but nope it takes the whole 1 hour for deps resolution [8].

The new resolver is known to take longer, however we are making it worse by having too many requirements there [7].

Perhaps one solution is to split those into separate jobs?

[1] https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020
[2] https://review.rdoproject.org/r/#/c/31358/9/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/requirements.txt
[3] https://review.rdoproject.org/r/#/c/31358/9/test-requirements.txt
[4] https://review.rdoproject.org/r/#/c/31358/9/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf/requirements.txt
[5] https://logserver.rdoproject.org/58/31358/8/check/tox-py36-ci-config/6d836f0/tox/py36-1.log
[6] http://paste.openstack.org/raw/801115/
[7] https://github.com/rdo-infra/ci-config/blob/c25934c03d15d317501bcf63ba2cdb737596f719/tox.ini#L44-L48
[8] https://logserver.rdoproject.org/58/31358/9/check/tox-py36-ci-config/cd11408/tox/py36-1.log