| 2019-02-12 10:27:26 |
Quique Llorente |
bug |
|
|
added bug |
| 2019-02-12 10:42:17 |
Quique Llorente |
description |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
Examples:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
standalone scn003
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_07_37_03
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
Examples:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
In the middle of this logs we can find
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}
standalone scn003
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_07_37_03
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
Same here
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:f1d2e3b7d076ab08269367d348980fda70468b45dfd2b82f4a713686f7582563"}]} |
|
| 2019-02-12 10:42:25 |
Quique Llorente |
summary |
Intermitent failures at tripleo-container-image-prepare at gates |
Intermitent failures at tripleo-container-image-prepare at gates with MANIFEST_BLOB_UNKNOWN |
|
| 2019-02-12 10:57:24 |
Quique Llorente |
description |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
Examples:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
In the middle of this logs we can find
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}
standalone scn003
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_07_37_03
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
Same here
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:f1d2e3b7d076ab08269367d348980fda70468b45dfd2b82f4a713686f7582563"}]} |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
In the middle of this logs we can find
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}
Also from journald http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/journal.txt.gz#_Feb_12_06_32_06
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=8cfa411d-904b-4c94-9025-9f6dbade0a64 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34328" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.428097ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=baffc051-6beb-475c-8cdf-ed4bfb23d2a0 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34360" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.245985ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1 err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=53196147-f8e7-4612-a74a-3bb7a8157ec8 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34362" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.851936ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1 HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=3aea2136-e3b8-422d-934c-1050f876c511 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34358" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.391882ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=cf6700b5-58e5-4130-b75d-1e567be4f75f http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34362" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.589816ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=768fc6f9-0ced-4714-992c-ee6960cbb22e http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34370" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.699994ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d HTTP/1.1" 404 157 "" "python-requests/2.19.1" |
|
| 2019-02-12 10:58:49 |
Quique Llorente |
description |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
In the middle of this logs we can find
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}
Also from journald http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/journal.txt.gz#_Feb_12_06_32_06
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=8cfa411d-904b-4c94-9025-9f6dbade0a64 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34328" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.428097ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=baffc051-6beb-475c-8cdf-ed4bfb23d2a0 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34360" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.245985ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e937483da5ef160543e2607fea7bebc8f903b6a9429588b7b36b824f094d6f7d HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1 err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=53196147-f8e7-4612-a74a-3bb7a8157ec8 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34362" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.851936ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:6c1fba0711e6f0a8f2d5da0835fb82136be093d15d88f3882020dbcca03ad5c1 HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=3aea2136-e3b8-422d-934c-1050f876c511 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34358" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.391882ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=cf6700b5-58e5-4130-b75d-1e567be4f75f http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34362" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=2.589816ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f HTTP/1.1" 404 157 "" "python-requests/2.19.1"
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=768fc6f9-0ced-4714-992c-ee6960cbb22e http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34370" http.request.uri="/v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.699994ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d" vars.name="tripleomaster/centos-binary-aodh-api" version="v2.6.2+unknown"
Feb 12 06:32:06 standalone.localdomain registry[20249]: 192.168.24.1 - - [12/Feb/2019:06:32:06 +0000] "HEAD /v2/tripleomaster/centos-binary-aodh-api/blobs/sha256:4ac91e6b2720f42ccbacebe7f2d2d285f99a17991f5448d956bc769f2e40470d HTTP/1.1" 404 157 "" "python-requests/2.19.1" |
Gate standalone jobs are starting to fail more ofthen at container preparation, but looking at the logs there is not much information:
standalone scn002
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_06_27_57
http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
In the middle of this logs we can find
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f"}]}
Also from journald http://logs.openstack.org/78/633278/14/gate/tripleo-ci-centos-7-scenario002-standalone/dca82b9/logs/undercloud/var/log/journal.txt.gz#_Feb_12_06_32_06
Feb 12 06:32:06 standalone.localdomain registry[20249]: time="2019-02-12T06:32:06Z" level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f err.message="blob unknown to registry" go.version=go1.9.4 http.request.host="192.168.24.1:8787" http.request.id=8cfa411d-904b-4c94-9025-9f6dbade0a64 http.request.method=HEAD http.request.remoteaddr="192.168.24.1:34328" http.request.uri="/v2/tripleomaster/centos-binary-aodh-notifier/blobs/sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" http.request.useragent="python-requests/2.19.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.428097ms http.response.status=404 http.response.written=157 instance.id=4071e315-a1b9-4d87-9b9b-11fc81da918b vars.digest="sha256:e7d3075c4205ccccbd4489430acc058e2958469a08829ae3906e65fd4da1551f" vars.name="tripleomaster/centos-binary-aodh-notifier" version="v2.6.2+unknown"
F
standalone scn003
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-02-12_07_37_03
http://logs.openstack.org/17/633617/10/gate/tripleo-ci-centos-7-scenario003-standalone/fae4151/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz
Same here
{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN","message":"blob unknown to registry","detail":"sha256:f1d2e3b7d076ab08269367d348980fda70468b45dfd2b82f4a713686f7582563"}]} |
|
| 2019-02-12 12:31:30 |
Emilien Macchi |
tripleo: milestone |
|
stein-3 |
|
| 2019-02-12 18:11:17 |
OpenStack Infra |
tripleo: status |
Triaged |
In Progress |
|
| 2019-02-12 18:11:17 |
OpenStack Infra |
tripleo: assignee |
Quique Llorente (quiquell) |
Emilien Macchi (emilienm) |
|
| 2019-02-12 18:15:21 |
Emilien Macchi |
tags |
alert |
alert ci containers |
|
| 2019-02-13 00:25:20 |
OpenStack Infra |
tripleo: status |
In Progress |
Fix Released |
|
| 2019-02-14 17:18:13 |
Emilien Macchi |
tripleo: status |
Fix Released |
In Progress |
|
| 2019-02-14 17:18:29 |
Emilien Macchi |
tags |
alert ci containers |
ci containers |
|
| 2019-02-20 21:02:27 |
OpenStack Infra |
tripleo: assignee |
Emilien Macchi (emilienm) |
Steve Baker (steve-stevebaker) |
|
| 2019-02-28 03:11:14 |
wes hayutin |
tags |
ci containers |
alert ci containers |
|
| 2019-03-11 19:34:31 |
OpenStack Infra |
tripleo: status |
In Progress |
Fix Released |
|
