RFE: (dlrnapi-promoter) Better handle Error removing image docker.io/tripleoqueens/centos-binary-nova-placement-api:current-tripleo - UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Bug #1771469 reported by Matt Young
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Invalid
High
Unassigned

Bug Description

A recent queens promotion log looks like this:

```
TASK [Delete the local images from docker hub]

<snip/>

failed: [localhost] (item=[u'nova-placement-api', u'current-tripleo']) => {"changed": false, "item": ["nova-placement-api", "current-tripleo"], "msg": "Error removing image docker.io/tripleoqueens/centos-binary-nova-placement-api:current-tripleo - UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)"}
```

This is happening here:

https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/container-push.yml#L170

We're getting a read timeout, and failing the entire promotion. In this case the image is actually deleted.

there's a few ways to handle this...

- We can not fail if we can't delete something. Allow promotion to continue (in this case to report results, etc).

- (better) We can leave images in the repository, then age them out intentionally. this will allow partial failure, and 10 mins later when we attempt to re-promote, we don't need to pay the > 1 hour tax of pulling ALL the container layers from rdoproject, and pushing ALL the layers to docker.io.

Tags: ci quickstart
Revision history for this message
Matt Young (halcyondude) wrote :

The full snip of the dlrnapi promoter script

Matt Young (halcyondude)
summary: - rfe: (dlrnapi-promoter) Better handle Error removing image
+ RFE: (dlrnapi-promoter) Better handle Error removing image
docker.io/tripleoqueens/centos-binary-nova-placement-api:current-tripleo
- UnixHTTPConnectionPool(host='localhost', port=None): Read timed out.
(read timeout=60)
Changed in tripleo:
milestone: rocky-2 → rocky-3
Changed in tripleo:
milestone: rocky-3 → rocky-rc1
Changed in tripleo:
milestone: rocky-rc1 → stein-1
Changed in tripleo:
milestone: stein-1 → stein-2
Changed in tripleo:
milestone: stein-2 → stein-3
Changed in tripleo:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.