All CentOS-8 jobs fail on missing various packages from component repos

Bug #1866543 reported by Sagi (Sergey) Shnaidman
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

2020-03-08 05:09:53 | [MIRROR] python3-barbicanclient-4.9.0-0.20191218125859.9c0e02d.el8.noarch.rpm: Status code: 403 for http://mirror.ord.rax.opendev.org:8080/rdo/centos8/component/clients/64/81/64814b9ace54e0151e9c28f4e57b87dafc984241_450aafcb/python3-barbicanclient-4.9.0-0.20191218125859.9c0e02d.el8.noarch.rpm
2020-03-08 05:09:53 | [FAILED] python3-barbicanclient-4.9.0-0.20191218125859.9c0e02d.el8.noarch.rpm: No more mirrors to try - All mirrors were already tried without success
2020-03-08 05:09:53 |
2020-03-08 05:09:53 | The downloaded packages were saved in cache until the next successful transaction.
2020-03-08 05:09:53 | You can remove cached packages by executing 'dnf clean packages'.
2020-03-08 05:09:53 | Error: Error downloading packages:
2020-03-08 05:09:53 | Cannot download python3-barbicanclient-4.9.0-0.20191218125859.9c0e02d.el8.noarch.rpm: All mirrors were tried

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e0a/711836/1/check/tripleo-ci-centos-8-containers-multinode/e0aca90/logs/undercloud/home/zuul/install_packages.sh.log

summary: - Standalone centos-8 job fails on missing package
+ Standalone centos-8 job fails on missing python3-barbicanclient
Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote : Re: Standalone centos-8 job fails on missing python3-barbicanclient

same error about python3-aodhclient package in tripleo-ci-centos-8-containers-multinode job:

https://0b50da5cfb0cec39d416-edc595b1b127a786093948ab304ce84b.ssl.cf2.rackcdn.com/695975/4/check/tripleo-ci-centos-8-containers-multinode/11d2ab6/logs/undercloud/home/zuul/install_packages.sh.log

2020-03-08 13:43:01 | [MIRROR] python3-aodhclient-1.5.0-0.20191218150343.428e394.el8.noarch.rpm: Status code: 403 for http://mirror.mtl01.inap.opendev.org:8080/rdo/centos8/component/clients/64/81/64814b9ace54e0151e9c28f4e57b87dafc984241_450aafcb/python3-aodhclient-1.5.0-0.20191218150343.428e394.el8.noarch.rpm
2020-03-08 13:43:01 | [FAILED] python3-aodhclient-1.5.0-0.20191218150343.428e394.el8.noarch.rpm: No more mirrors to try - All mirrors were already tried without success
2020-03-08 13:43:01 |
2020-03-08 13:43:01 | The downloaded packages were saved in cache until the next successful transaction.
2020-03-08 13:43:01 | You can remove cached packages by executing 'dnf clean packages'.
2020-03-08 13:43:01 | Error: Error downloading packages:
2020-03-08 13:43:01 | Cannot download python3-aodhclient-1.5.0-0.20191218150343.428e394.el8.noarch.rpm: All mirrors were tried

summary: - Standalone centos-8 job fails on missing python3-barbicanclient
+ Standalone centos-8 job fails on missing packages
summary: - Standalone centos-8 job fails on missing packages
+ Standalone centos-8 job fails on missing various packages
summary: - Standalone centos-8 job fails on missing various packages
+ CentOS-8 job fail on missing various packages
Revision history for this message
Sagi (Sergey) Shnaidman (sshnaidm) wrote :

seems like need to whitelist component repos in pruning script:
https://github.com/rdo-infra/puppet-dlrn/blob/master/files/run-purge.sh#L30

summary: - CentOS-8 job fail on missing various packages
+ All CentOS-8 jobs fail on missing various packages
summary: - All CentOS-8 jobs fail on missing various packages
+ All CentOS-8 jobs fail on missing various packages from component repos
Revision history for this message
Javier Peña (jpena-c) wrote :

We added code to DLRN so it would treat component-based repos correctly in dlrn-purge, even if all symlinks were not defined there (just need to define the high-level ones, like promoted-components)[1].

The issue here is that, since promotions are done in the public-facing server, but purges are done in the non-public one, we need to sync all promotion-related symlinks. We did that in the past, but needed to fix that to include component support. [2] will fix it.

[1] - https://softwarefactory-project.io/r/17638
[2] - https://review.rdoproject.org/r/25770

Revision history for this message
Rabi Mishra (rabi) wrote :

I guess [2] has merged sometime back. Just noticed @https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_078/711838/2/check/tripleo-build-containers-centos-8/0783331/logs/buildah-builds/kolla-builds/32f20fff-323a-4811-8af5-3167857590a9/docker/base/base-build.log

[FAILED] puppet-archive-4.4.1-0.20191222001801.810fd93.el8.noarch.rpm: No more mirrors to try - All mirrors were already tried without success

Would it take some more time or additional promotions?

Revision history for this message
Javier Peña (jpena-c) wrote :

We need additional promotions, since the deleted packages cannot be recovered.

Revision history for this message
Alan Pevec (apevec) wrote :

In theory, we have commits of deleted packages in dlrn db and _could_ rebuild them?

Revision history for this message
wes hayutin (weshayutin) wrote :

ok.. so upstream check and gate were fixed.
rlandy and I had to promote consistent to tripleo-ci-testing to fix integration jobs.

done done..

Changed in tripleo:
status: Triaged → Fix Released
Revision history for this message
wes hayutin (weshayutin) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.