centos7-rt repo is not mirrored with nodepool

Bug #1814872 reported by Quique Llorente on 2019-02-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
Quique Llorente

Bug Description

http://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/4691689/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz
019-02-06 05:55:32 | 2019-02-06 05:55:32.440 | Trying other mirror.
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | One of the configured repositories failed (CentOS 7 - Realtime),
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | and yum doesn't have enough cached data to continue. At this point the only
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | safe thing yum can do is fail. There are a few ways to work "fix" this:
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | 1. Contact the upstream for the repository and get them to fix the problem.
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | 2. Reconfigure the baseurl/etc. for the repository, to point to a working
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | upstream. This is most often useful if you are using a newer
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | distribution release than is supported by the repository (and the
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | packages for the previous distribution release still work).
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | 3. Run the command with the repository temporarily disabled
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | yum --disablerepo=centos7-rt ...
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | 4. Disable the repository permanently, so yum won't use it by default. Yum
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | will then just ignore the repository until you permanently enable it
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | again or use --enablerepo for temporary usage:
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | yum-config-manager --disable centos7-rt
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | or
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | subscription-manager repos --disable=centos7-rt
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | 5. Configure the failing repository to be skipped, if it is unavailable.
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | Note that yum will try to contact the repo. when it runs most commands,
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | so will have to try and fail each time (and thus. yum will be be much
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | slower). If it is a very temporary problem though, this is often a nice
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | compromise:
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | yum-config-manager --save --setopt=centos7-rt.skip_if_unavailable=true
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 |
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | failure: repodata/repomd.xml from centos7-rt: [Errno 256] No more mirrors to try.
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:32 | 2019-02-06 05:55:32.444 | http://mirror.centos.org/centos/7/rt/x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2607:f128:40:1600:225:90ff:fe00:bde6: Network is unreachable"
2019-02-06 05:55:35 | 2019-02-06 05:55:35.029 | INFO diskimage_builder.block_device.blockdevice [-] State already cleaned - no way to do anything here
2019-02-06 05:55:35 | 2019-02-06 05:55:35.129 | Unmount /tmp/dib_build.LmDHcPmg/mnt/tmp/yum
2019-02-06 05:55:35 | 2019-02-06 05:55:35.196 | Unmount /tmp/dib_build.LmDHcPmg/mnt/tmp/in_target.d
2019-02-06 05:55:35 | 2019-02-06 05:55:35.230 | Unmount /tmp/dib_build.LmDHcPmg/mnt/sys

After this [1] there is a new repo centos7-rt at build-image in case we need it we have to mirror it to nodepool mirrors.

[1] https://github.com/openstack/tripleo-puppet-elements/commit/120eca81d864d709c470dd651a23e08bf10967a1

Quique Llorente (quiquell) wrote :

Looks like we have to use centos7-rt mirror at RDO jobs.

11:54 <owalsh> quiquell|rover: hrmm, yea, one of the packages is installed on the default images -tuned_profiles_cpu_partitioning_package in https://github.com/openstack/tripleo-puppet-elements/blob/master/elements/overcloud-compute/install.d/package-installs-overcloud-compute

summary: - queens: centos7-rt repo is not mirrored with nodepool
+ centos7-rt repo is not mirrored with nodepool

Fix proposed to branch: master
Review: https://review.openstack.org/635144

Changed in tripleo:
status: Triaged → In Progress

Change abandoned by Quique Llorente (<email address hidden>) on branch: master
Review: https://review.openstack.org/635144

Reviewed: https://review.openstack.org/635135
Committed: https://git.openstack.org/cgit/openstack/tripleo-puppet-elements/commit/?id=d313a060c80f5d97c49092eb695c335be102b473
Submitter: Zuul
Branch: master

commit d313a060c80f5d97c49092eb695c335be102b473
Author: Quique Llorente <email address hidden>
Date: Wed Feb 6 12:38:42 2019 +0100

    Use external centos7-rt if present

    Depending on the environment we execute this we need different
    configuration for the centos7-rt repo, like at RDO zuul ci we have to
    use nodepool mirrors, this patch check for a centos7-rt repo and enables
    it if it's not present just creates a default one

    Change-Id: I630a399d95752b1fb1e30220a582ace8d7b97f18
    Related-Bug: #1814872

Reviewed: https://review.openstack.org/636277
Committed: https://git.openstack.org/cgit/openstack/tripleo-puppet-elements/commit/?id=90370d4c943616d1f3f976b5257f48caff1f55af
Submitter: Zuul
Branch: stable/rocky

commit 90370d4c943616d1f3f976b5257f48caff1f55af
Author: Quique Llorente <email address hidden>
Date: Wed Feb 6 12:38:42 2019 +0100

    Use external centos7-rt if present

    Depending on the environment we execute this we need different
    configuration for the centos7-rt repo, like at RDO zuul ci we have to
    use nodepool mirrors, this patch check for a centos7-rt repo and enables
    it if it's not present just creates a default one

    Change-Id: I630a399d95752b1fb1e30220a582ace8d7b97f18
    Related-Bug: #1814872
    (cherry picked from commit d313a060c80f5d97c49092eb695c335be102b473)

tags: added: in-stable-rocky
tags: added: in-stable-queens

Reviewed: https://review.openstack.org/636278
Committed: https://git.openstack.org/cgit/openstack/tripleo-puppet-elements/commit/?id=c6dc9fbbc6189d2c1113de830da45650f6951cc7
Submitter: Zuul
Branch: stable/queens

commit c6dc9fbbc6189d2c1113de830da45650f6951cc7
Author: Quique Llorente <email address hidden>
Date: Wed Feb 6 12:38:42 2019 +0100

    Use external centos7-rt if present

    Depending on the environment we execute this we need different
    configuration for the centos7-rt repo, like at RDO zuul ci we have to
    use nodepool mirrors, this patch check for a centos7-rt repo and enables
    it if it's not present just creates a default one

    Change-Id: I630a399d95752b1fb1e30220a582ace8d7b97f18
    Related-Bug: #1814872
    (cherry picked from commit d313a060c80f5d97c49092eb695c335be102b473)

tags: added: in-stable-pike

Reviewed: https://review.openstack.org/636289
Committed: https://git.openstack.org/cgit/openstack/tripleo-puppet-elements/commit/?id=5a6f6dbb8d3199888cc124e5fe6c7a4e0a3b5502
Submitter: Zuul
Branch: stable/pike

commit 5a6f6dbb8d3199888cc124e5fe6c7a4e0a3b5502
Author: Quique Llorente <email address hidden>
Date: Tue Feb 12 10:38:37 2019 +0100

    Use external centos7-rt if present

    Depending on the environment we execute this we need different
    configuration for the centos7-rt repo, like at RDO zuul ci we have to
    use nodepool mirrors, this patch check for a centos7-rt repo and enables
    it if it's not present just creates a default one

    Change-Id: I630a399d95752b1fb1e30220a582ace8d7b97f18
    Related-Bug: #1814872
    (cherry picked from commit d313a060c80f5d97c49092eb695c335be102b473)

Reviewed: https://review.openstack.org/635157
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=331fa892fa5a050c42bdd2edd299fdd343367615
Submitter: Zuul
Branch: master

commit 331fa892fa5a050c42bdd2edd299fdd343367615
Author: Quique Llorente <email address hidden>
Date: Wed Feb 6 14:43:28 2019 +0100

    Add centos7-rt repository

    The centos-rt was added for tuned profile cpu-partitioning as part of [1] now
    one of the package is installed in the default image so we generate the
    proper disabled repo now with nodepool mirrors and it will get copied to
    overcloud image, then the dib element will enable it
    master is merged [2]

    Not added as depend on since this review works without them
    tripleo-puppet-element changes

    master: https://review.openstack.org/635135
    rocky: https://review.openstack.org/#/c/636277/
    queens: https://review.openstack.org/#/c/636278/
    pike: https://review.openstack.org/#/c/636289/
    ocata: https://review.openstack.org/#/c/636290/

    Closes-Bug: #1814872

    [1] https://review.openstack.org/454732,

    Change-Id: Ie2ef9b836d8bc3a701dbb1bad907263d17d2b5d7

Changed in tripleo:
status: In Progress → Fix Released

Reviewed: https://review.openstack.org/636290
Committed: https://git.openstack.org/cgit/openstack/tripleo-puppet-elements/commit/?id=5a94f37e426e78225a40b6ee850989625c1399ed
Submitter: Zuul
Branch: stable/ocata

commit 5a94f37e426e78225a40b6ee850989625c1399ed
Author: Quique Llorente <email address hidden>
Date: Tue Feb 12 10:38:37 2019 +0100

    Use external centos7-rt if present

    Depending on the environment we execute this we need different
    configuration for the centos7-rt repo, like at RDO zuul ci we have to
    use nodepool mirrors, this patch check for a centos7-rt repo and enables
    it if it's not present just creates a default one

    Change-Id: I630a399d95752b1fb1e30220a582ace8d7b97f18
    Related-Bug: #1814872
    (cherry picked from commit d313a060c80f5d97c49092eb695c335be102b473)

tags: added: in-stable-ocata
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers