OVB jobs on c9 are hitting RETRY status - Max retries exceeded with url: /openstack/requirements/stable/wallaby/upper-constraints.txt

Bug #1990556 reported by Ronelle Landy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

OVB jobs on c9 are hitting RETRY status:

2022-09-22 15:50:46.073454 | TASK [ovb-manage : Install openstack clients in venv]
2022-09-22 15:57:32.388322 | primary | ERROR
2022-09-22 15:57:32.388577 | primary | {
2022-09-22 15:57:32.388613 | primary | "attempts": 3,
2022-09-22 15:57:32.388639 | primary | "msg": "\n:stderr: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c7db20>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /openstack/requirements/stable/wallaby/upper-constraints.txt\nWARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c7dd60>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /openstack/requirements/stable/wallaby/upper-constraints.txt\nWARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c7df40>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /openstack/requirements/stable/wallaby/upper-constraints.txt\nWARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c24160>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /openstack/requirements/stable/wallaby/upper-constraints.txt\nWARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c24340>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /openstack/requirements/stable/wallaby/upper-constraints.txt\nERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /openstack/requirements/stable/wallaby/upper-constraints.txt (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa5c9c24520>: Failed to establish a new connection: [Errno 101] Network is unreachable'))\n\nWARNING: You are using pip version 21.2.3; however, version 22.2.2 is available.\nYou should consider upgrading via the '/home/zuul/workspace/ovb/.venv/bin/python3 -m pip install --upgrade pip' command.\n"
2022-09-22 15:57:32.388676 | primary | }

The history on RETRY status is:
https://review.rdoproject.org/zuul/builds?result=RETRY&skip=0

The error is coming from this task:
https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/roles/ovb-manage/tasks/install_openstack_deps.yml#L25

https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/roles/ovb-manage/vars/centos-9.yml#L16
constraint_file: https://raw.githubusercontent.com/openstack/requirements/stable/wallaby/upper-constraints.txt

Revision history for this message
Ronelle Landy (rlandy) wrote :

Marking promotion-blocker as it hits lines.

Suggestion is to change to: https://opendev.org/openstack/requirements/raw/branch/stable/wallaby/upper-constraints.txt url for constriants

Changed in tripleo:
milestone: none → zed-1
importance: Undecided → Critical
status: New → Triaged
tags: added: promotion-blocker
Revision history for this message
chandan kumar (chkumar246) wrote :

https://review.rdoproject.org/r/c/config/+/45232 Replace constraint_file link got merged.

Revision history for this message
chandan kumar (chkumar246) wrote :
Revision history for this message
Ananya Banerjee (frenzyfriday) wrote :
Revision history for this message
chandan kumar (chkumar246) wrote :

In order to further resolve this issue, we are doing following :

- 45286: Use Vexxhost DNS local nameserver for pre playbook | https://review.rdoproject.org/r/c/rdo-jobs/+/45286
- 45296: [ovb] Add openstack/requirements under required-projects | https://review.rdoproject.org/r/c/rdo-jobs/+/45296
- 45304: Use upper-constraints.txt from clonned openstack/requirements | https://review.rdoproject.org/r/c/config/+/45304

Once the above patches merges, we can monitor the jobs.

Revision history for this message
Ananya Banerjee (frenzyfriday) wrote :

After the fixes were merged we don't see failed retries for upper-constraints.txt
Maybe we can close this EOD

Alan Pevec (apevec)
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.