RHEL8 standalone check jobs are failing to find containers tagged w/o arch in rdoproject registry

[Ronelle Landy]

All rhel-8 standalone check jobs are failing with the following error:

tripleo_common.image.exception.ImageNotFoundException: Not found image:docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582

The tag is now 19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582_x86_64 in trunk.registry.rdoproject.org. dockerhub has https://hub.docker.com/r/tripleomaster/rhel-binary-aodh-api/tags 19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582

See list of failing jobs in https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-rhel-8-scenario001-standalone-rdo

Possibly due to the new reproducer testing as the old reroducer has rhel8 disabled.
Possibly a duplicate of https://bugs.launchpad.net/tripleo/+bug/1850527

[Rafael Folco (rafaelfolco)]

This is broken for multiple reasons I think.

- promotion is not atomic and its currently broken on manifest push so hashes update happen after that point might be causing out-of-sync containers on check jobs - https://review.rdoproject.org/r/23582

- rhel is pulling from rdoproject registry while centos jobs pull from docker.io
(Credits to @rlandy) Can you please add your findings to this bug?

Thanks.

[Marios Andreou (marios-b)]

not clear what the problem is... even though the points in comment #1 are correct ...

we should file specific bug when we have concrete errror to fix not clear here moved invalid