Description
===========
Containers are failed to upload in undercloud local registry
Affected Jobs
=============
- tripleo-ci-centos-8-undercloud-upgrade-ussuri
Logs
====
- https://18e9d4504dfedf09309f-a7117e65d5d46fb2ebde9a8b3aa13b86.ssl.cf5.rackcdn.com/789405/1/check/tripleo-ci-centos-8-undercloud-upgrade-ussuri/6f99267/logs/undercloud/home/zuul/undercloud_install.log
- https://2319feb14e831f87c231-4854ed941a7816d1225c43ae9b456d0e.ssl.cf1.rackcdn.com/788372/1/gate/tripleo-ci-centos-8-undercloud-upgrade-ussuri/913ab62/logs/undercloud/home/zuul/undercloud_install.log
Actual Error
============
{'image_source': 'tripleo', 'imagename': '10.4.70.136:5001/tripleotraincentos8/centos-binary-zaqar-wsgi:d943755a88f99a2f945134303d31109e', 'push_destination': '192.168.24.1:8787'}]\nDEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): 10.4.70.136:5001\nINFO:tripleo_common.image.image_uploader:[10.4.70.136:5001/tripleotraincentos8/centos-binary-glance-api:d943755a88f99a2f945134303d31109e] Starting upload image process\nINFO:tripleo_common.image.image_uploader:[10.4.70.136:5001/tripleotraincentos8/centos-binary-ironic-inspector:d943755a88f99a2f945134303d31109e] Starting upload image process\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 192.168.24.1:8787\nINFO:tripleo_common.image.image_uploader:[10.4.70.136:5001/tripleotraincentos8/centos-binary-zaqar-wsgi:d943755a88f99a2f945134303d31109e] Starting upload image process\nDEBUG:urllib3.connectionpool:http://192.168.24.1:8787 \"GET /v2/ HTTP/1.1\" 200 2\nINFO:tripleo_common.image.image_uploader:[10.4.70.136:5001/tripleotraincentos8/centos-binary-swift-account:d943755a88f99a2f945134303d31109e] Starting upload image process\nDEBUG:tripleo_common.image.image_uploader:http://192.168.24.1:8787/v2/ status code 200\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.util.retry:Converted retries value: 8 -> Retry(total=8, connect=None, read=None, redirect=None, status=None)\nDEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 192.168.24.1:8787\nDEBUG:urllib3.connectionpool:http://192.168.24.1:8787 \"POST /v2/tripleotraincentos8/centos-binary-ironic-inspector/blobs/uploads/ HTTP/1.1\" 404 196\nINFO:tripleo_common.image.image_uploader:Non-2xx: id 718484d7e74eb159eedb7b4fcd176364458d09bb, status 404, reason Not Found, text <!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p>The requested URL was not found on this server.</p>\n</body></html>\n
akahat|ruck> marios, hey.. undercloud-
13:04 < akahat|ruck> i thought you might know about it. I've open bz for it ^^
o/ hello.
so the problem isn't the pushing to local registry but the problem is pulling from the content provider.
We've seen it before (I think one of the clouds in particular had issues with that) but it seems to be transient.
In the case of [1] the problem is with the train content provider:
* 10.4.70.136 timed out. (connect timeout=30)')': /v2/\nDEBUG:
and we can check that this is the train content provider from the inventory [2]
* ansible_host: 104.130.198.52
* provider_
registry_
train: 10.4.70.136
ussuri: 23.253.213.48
So if it keeps happening (usually it doesn't) you may want to see if it is consistently a problem with the train provider and see what's up if anything there. It could be a network/infra issue with the cloud provider
[1] https:/
[2] https:/