[master] [victoria] [ussuri] Errors during downloading metadata for repository 'quickstart-centos-appstreams'

[Bhagyashri Shewale]

2021-03-12 06:16:07.936552 | primary | 2021-03-12 06:16:03 | opstools 2.5 MB/s | 148 kB 00:00
2021-03-12 06:16:07.936571 | primary | 2021-03-12 06:16:03 | NFV SIG OpenvSwitch 39 kB/s | 29 kB 00:00
2021-03-12 06:16:07.936602 | primary | 2021-03-12 06:16:05 | centos-8-fix 15 kB/s | 19 kB 00:01
2021-03-12 06:16:07.936625 | primary | 2021-03-12 06:16:07 | quickstart-centos-appstreams 30 kB/s | 78 kB 00:02
2021-03-12 06:16:07.936645 | primary | 2021-03-12 06:16:07 | Errors during downloading metadata for repository 'quickstart-centos-appstreams':
2021-03-12 06:16:07.936665 | primary | 2021-03-12 06:16:07 | - Status code: 404 for http://mirror.mtl01.inap.opendev.org/centos/8-stream/AppStream/x86_64/os/repodata/07079f39e840ac2bf2f8aee5ff79aade080508502fdb482f51bd16542632b34c-primary.xml.gz (IP: 198.72.125.6)
2021-03-12 06:16:07.936699 | primary | 2021-03-12 06:16:07 | - Status code: 404 for http://mirror.mtl01.inap.opendev.org/centos/8-stream/AppStream/x86_64/os/repodata/3f11c4b4e5140028629dbd789f78c7bd365c670323eaa757d12a092b089be291-filelists.xml.gz (IP: 198.72.125.6)
2021-03-12 06:16:07.936729 | primary | 2021-03-12 06:16:07 | Error: Failed to download metadata for repo 'quickstart-centos-appstreams': Yum repo downloading error: Downloading error(s): repodata/07079f39e840ac2bf2f8aee5ff79aade080508502fdb482f51bd16542632b34c-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/3f11c4b4e5140028629dbd789f78c7bd365c670323eaa757d12a092b089be291-filelists.xml.gz - Cannot download, all mirrors were already tried without success; repodata/5bc0a4b682916a8fbbcc10ed5d8aece7bac1213bfd73aa9452cda17bb05000f2-modules.yaml.gz - Cannot download, all mirrors were already tried without success
2021-03-12 06:16:07.936755 | primary |
2021-03-12 06:16:07.936775 | primary |
2021-03-12 06:16:07.936794 | primary | MSG:
2021-03-12 06:16:07.936812 | primary |
2021-03-12 06:16:07.936831 | primary | non-zero return code
2021-03-12 06:16:07.936850 | primary |
2021-03-12 06:16:07.936869 | primary | PLAY RECAP *********************************************************************
2021-03-12 06:16:07.936888 | primary | undercloud : ok=37 changed=14 unreachable=0 failed=1 skipped=39 rescued=0 ignored=0
2021-03-12 06:16:07.936907 | primary |
2021-03-12 06:16:07.936925 | primary | Friday 12 March 2021 06:16:07 +0000 (0:00:15.926) 0:01:06.952 **********
2021-03-12 06:16:07.936944 | primary | ===============================================================================

#link:

1. https://ecc955bb769bab7a07be-c205db93b3c29bdefc78a96a38b9c4c7.ssl.cf1.rackcdn.com/periodic/opendev.org/x/tobiko/master/tripleo-ci-centos-8-content-provider/a2d6f7c/job-output.txt

2. https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1cc/779799/1/check/tripleo-ci-centos-8-content-provider-ussuri/1cc9190/job-output.txt

3. https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8da/779799/1/check/tripleo-ci-centos-8-content-provider-victoria/8dabad7/job-output.txt

4. https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8da/779799/1/check/tripleo-ci-centos-8-content-provider-victoria/8dabad7/job-output.txt

5. https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-content-provider&job_name=tripleo-ci-centos-8-content-provider-ussuri&job_name=tripleo-ci-centos-8-content-provider-victoria

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi All,

FYI,

<ykarel> ianw, hi, u around?
<ykarel> we facing mirror issues for centos 8-stream, and on seeing i see the centos mirror from which infra mirrors follow is not synched for 12 hours
<ykarel> http://mirror.dal10.us.leaseweb.net/centos/8-stream/AppStream/x86_64/os/repodata/ one
<ykarel> in https://mirror-status.centos.org/ i see some mirror which are good
<openstackgerrit> Merged opendev/system-config master: refstack: add backup https://review.opendev.org/c/opendev/system-config/+/775061
<ykarel> the mirror u added in https://review.opendev.org/c/opendev/system-config/+/684437 is good currently, which was changed later to current one ^ as that was not up to date that time and was not in mirror-status.centos,org
<ykarel> in https://review.opendev.org/c/opendev/system-config/+/716602

<ykarel> ok http://mirror.dal10.us.leaseweb.net/centos/8-stream/AppStream/x86_64/os/repodata/ is updated now, so next rsync should fix infra mirrors

<ykarel> last run missed that, and now next run is in approx 1.25 hour
<ykarel> if it can be manually triggered before that it will be good, else have to wait

<ykarel> mirrors got updated now

<ianw> ykarel: sorry, missed this, things in sync now?
<ykarel> ianw, yes now it's synched
<ykarel> ianw, now seeing issue with epel repos not synched

<ykarel> http://pubmirror1.math.uh.edu/fedora-buffet/epel/8/Everything/x86_64/repodata/?C=M;O=D vs mirror.ord.rax.opendev.org/epel/8/Everything/x86_64/repodata/?C=M;O=D
<ykarel> and other epel mirror https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/repodata/?C=M;O=D

[Alan Pevec (apevec)]

This was resolved by opendev infra team, please close.