I think there is a bug in the python uploader, where a blob would be missing:
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=8cfa411d-904b-4c94-9025-9f6dbade0a64 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34328" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.428097ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown" Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f HTTP/1.1" 404 157 "" "python-requests/2.19.1"
It returns 404 which I don't think is considered as a requests.exceptions.RequestException. If that's the case, the _cross_repo_mount won't retry: https://git.openstack.org/cgit/openstack/tripleo-common/tree/tripleo_common/image/image_uploader.py#n579
And move on to the next layer, and at the end report a failure.
I think there is a bug in the python uploader, where a blob would be missing:
Feb 12 06:32:06 standalone. localdomain registry[20249]: time="2019- 02-12T06: 32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail= sha256: e7d3075c4205ccc cbd4489430acc05 8e2958469a08829 ae3906e65fd4da1 551f err.message="blob unknown to registry" go.version=go1.9.4 http.request. host="192. 168.24. 1:8787" http.request. id=8cfa411d- 904b-4c94- 9025-9f6dbade0a 64 http.request. method= HEAD http.request. remoteaddr= "192.168. 24.1:34328" http.request. uri="/v2/ tripleomaster/ centos- binary- aodh-notifier/ blobs/sha256: e7d3075c4205ccc cbd4489430acc05 8e2958469a08829 ae3906e65fd4da1 551f" http.request. useragent= "python- requests/ 2.19.1" http.response. contenttype= "application/ json; charset=utf-8" http.response. duration= 3.428097ms http.response. status= 404 http.response. written= 157 instance. id=4071e315- a1b9-4d87- 9b9b-11fc81da91 8b vars.digest= "sha256: e7d3075c4205ccc cbd4489430acc05 8e2958469a08829 ae3906e65fd4da1 551f" vars.name= "tripleomaster/ centos- binary- aodh-notifier" version= "v2.6.2+ unknown" localdomain registry[20249]: 192.168.24.1 - - [12/Feb/ 2019:06: 32:06 +0000] "HEAD /v2/tripleomast er/centos- binary- aodh-notifier/ blobs/sha256: e7d3075c4205ccc cbd4489430acc05 8e2958469a08829 ae3906e65fd4da1 551f HTTP/1.1" 404 157 "" "python- requests/ 2.19.1"
Feb 12 06:32:06 standalone.
It returns 404 which I don't think is considered as a requests. exceptions. RequestExceptio n. /git.openstack. org/cgit/ openstack/ tripleo- common/ tree/tripleo_ common/ image/image_ uploader. py#n579
If that's the case, the _cross_repo_mount won't retry:
https:/
And move on to the next layer, and at the end report a failure.