Track pip package download retries (note this often does not cause jobs to fail)

Bug #1708686 reported by Clark Boylan on 2017-08-04
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack-Gate
Undecided
Sorin Sbarnea

Bug Description

Pip will make up to 5 attempts to download a package. If it fails you get log messages like:

2017-08-04 13:43:05.623 | Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='mirror.lon1.citycloud.openstack.org', port=8080): Read timed out. (read timeout=60.0)",)': /pypi/simple/sphinxcontrib-pecanwsme/

Because we will then retry 4 more times after this failure chances are that we eventually succeed. This means this error is often not fatal. But we want to track occurences of this as it may point to larger underlying network issues.

This bug is being created to track this specific mostly not fatal problem so that it is clear when looking at the elastic-recheck list that this bug while common is 1) likely not the cause of the actual failure so please do look into it more and 2) is not as high a priority as the bugs that actually cause failures per 1).

Sorin Sbarnea (ssbarnea) on 2018-11-29
Changed in openstack-gate:
status: New → In Progress
assignee: nobody → Sorin Sbarnea (ssbarnea)
Sorin Sbarnea (ssbarnea) wrote :

We are currently using local mirrors in most places which accelerates and lower the chance of getting an error but even these do fail from time to time, causing lots of build failures.

Due to this I am working and implementing a fallback mecanism what should make pip look first at local mirror but also try to use official mirrors, so temporary glitches would not affect the builds.

One such change is at https://launchpad.net/bugs/1708686 but we need to test it better as there are some known side effects on how this may affect pip behavior.

Related:
* https://github.com/pypa/pip/issues/6045

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.