Stein branch check/Gate jobs are failing because of missing container images, Error - ImageNotFoundException

[Sandeep Yadav]

Descriptions:-

Stein check/Gate jobs are failing because of missing container images

Logs snippet:-

https://149508874915607fdcac-8ec2ce4c808d48e5dc938b34ec8c6e3a.ssl.cf2.rackcdn.com/774581/2/check/tripleo-ci-centos-7-scenario001-standalone/724872e/logs/undercloud/home/zuul/standalone_deploy.log
~~~
ImageNotFoundException: Not found image: docker://quay.io/tripleostein/centos-binary-aodh-api:237fe69c85441a323af53084ac3380306cbe42f4_8d9f1861
~~~

https://6ffa64e19007d190d535-a51f273e08e05c861a2fa6ace8476176.ssl.cf2.rackcdn.com/774581/2/check/tripleo-ci-centos-7-scenario004-standalone/482dc7c/logs/undercloud/home/zuul/standalone_deploy.log

~~~
ImageNotFoundException: Not found image: docker://quay.io/tripleostein/centos-binary-cron:237fe69c85441a323af53084ac3380306cbe42f4_8d9f1861
~~~

Indeed there is no 237fe69c85441a323af53084ac3380306cbe42f4_8d9f1861 @ https://quay.io/repository/tripleostein/centos-binary-aodh-api?tab=tags

Even though this is the valid current-tripleo @ https://trunk.rdoproject.org/centos7-stein/current-tripleo/delorean.repo ==> baseurl=https://trunk.rdoproject.org/centos7-stein/23/7f/237fe69c85441a323af53084ac3380306cbe42f4_8d9f1861

This hash was promoted on 13th Feb.

[Marios Andreou (marios-b)]

this is blocking https://review.opendev.org/c/openstack/paunch/+/774581 as commented there https://review.opendev.org/c/openstack/paunch/+/774581/2//COMMIT_MSG#9

[Sandeep Yadav (sandeepyadav93)]

We got current tripleo promotion on 17th feb, But containers with tag were pushed to docker not quay:-

[zuul@undercloud ~]$ podman pull docker.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4
Trying to pull docker.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4...
Getting image source signatures
Copying blob 2d473b07cdd5 skipped: already exists
Copying blob c214642c07e7 done
Copying blob 2a24e620be89 done
Copying blob 8ad80cb58191 done
Copying blob 35d94162082c done
Copying blob ed194261bff1 done
Copying blob 8c9bd50e5dad done
Copying blob 88378e66a951 done
Copying blob 81b5b3ffd322 done
Copying blob 46fbdf70981a done
Copying blob 29e21fa1ae53 done
Copying blob 477e63b6de70 done
Copying blob 47e74a42823b done
Copying blob 0c59583305ac done
Copying blob 439a31edb111 done
Copying blob 564fbfd9cfbc done
Copying blob 9252e4d5b813 done
Copying blob 4e326ec6bce2 done
Copying blob de026e2a4ab5 done
Copying blob 609ed6b2b10c done
Copying blob ecd114ec5d7e done
Copying blob 41c150c22613 done
Copying blob 32e8e288809a done
Copying blob 9759e7cd6801 done
Copying blob 15e3ab1d6d16 done
Copying blob 54553ee16a81 done
Copying blob 8b736a68b5a4 done
Copying blob 65d389356e9e done
Copying blob e8520782d6ad done
Copying blob 498bba794fc5 done
Copying config 7f495dd286 done
Writing manifest to image destination
Storing signatures
7f495dd286631b9201e3e235313d8bcb653bef62e9c64faba51797dfa93d4374

[zuul@undercloud ~]$ podman pull quay.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4
Trying to pull quay.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4...
  manifest unknown: manifest unknown
Error: error pulling image "quay.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4": unable to pull quay.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4: unable to pull image: Error initializing source docker://quay.io/tripleostein/centos-binary-nova-api:f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4: Error reading manifest f60ad6c2010e9a65a993fbda9a35bf2a656c73d7_893d6aa4 in quay.io/tripleostein/centos-binary-nova-api: manifest unknown: manifest unknow

our jobs is looking for "namespace: quay.io/tripleostein"

https://13b50ff454f43e41fea1-70aee045aa856a76767e0cd0433cf359.ssl.cf5.rackcdn.com/774581/2/check/tripleo-ci-centos-7-scenario001-standalone/71d6c8e/logs/undercloud/home/zuul/containers-prepare-parameters.yaml

~~~
namespace: quay.io/tripleostein
~~~

[wes hayutin (weshayutin)]

current-tripleo 2 hours ago
2 Medium
471.2 MB Never SHA256 bf2fe14232c0
903834017d713b34a615aeb380c0dc10aafb71c6_8d9f1861 9 hours ago
2 Medium
471.2 MB Never SHA256 bf2fe14232c0

[wes hayutin (weshayutin)]

@arx stein looks good, can this be closed?

[Sandeep Yadav (sandeepyadav93)]

Stein check jobs are still failing because of missing containers:-

https://376fd4347bf0b76b784e-9c39e1b31aead2b89889cdc7bed43508.ssl.cf5.rackcdn.com/774581/2/check/tripleo-ci-centos-7-scenario001-standalone/959f385/logs/undercloud/home/zuul/standalone_deploy.log

[Ronelle Landy (rlandy)]

we don;t watch stein any more