victoria updates fail w/ Problem: package libknet1-compress-bzip2-plugin-1.16-1.el8.x86_64 requires libknet1(x86-64) = 1.16-1.el8, but none of the providers can be installed

[wes hayutin]

     FATAL | Update all packages | centos-8-rax-iad-0023396808 | error={"changed": false, "failures": [], "msg": "Depsolve Error occured: \n Problem: package libknet1-compress-bzip2-plugin-1.16-1.el8.x86_64 requires libknet1(x86-64) = 1.16-1.el8, but none of the providers can be installed\n - cannot install both libknet1-1.18-1.el8.x86_64 and libknet1-1.16-1.el8.x86_64\n - cannot install both libknet1-1.16-1.el8.x86_64 and libknet1-1.18-1.el8.x86_64\n - cannot install the best update candidate for package libknet1-compress-bzip2-plugin-1.16-1.el8.x86_64\n - cannot install the best update candidate for package libknet1-1.16-1.el8.x86_64", "rc": 1, "results": []}

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_972/778832/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-victoria/9727ed1/logs/undercloud/home/zuul/overcloud_update_run-Controller.log

[wes hayutin (weshayutin)]

libknet1.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-bzip2-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-lz4-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-lzma-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-lzo2-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-plugins-all.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-compress-zlib-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-crypto-nss-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-crypto-openssl-plugin.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-crypto-plugins-all.x86_64 1.16-1.el8 @quickstart-centos-highavailability
libknet1-plugins-all.x86_64 1.16-1.el8 @quickstart-centos-highavailability

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_972/778832/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-victoria/9727ed1/logs/subnode-1/var/log/extra/package-list-installed.txt

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_972/778832/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-victoria/9727ed1/logs/subnode-1/etc/yum.repos.d/quickstart-centos-highavailability.repo

not sure where libknet-1.18 is coming from

[wes hayutin (weshayutin)]

This is resetting the gate... upgrade jobs may need to go non-voting if we can't find why upgrade is looking for 1.18 vs. 1.16

[wes hayutin (weshayutin)]

PowerTools is also at 1.16
http://mirror.mtl01.inap.opendev.org/centos/8/PowerTools/x86_64/os/Packages/

[wes hayutin (weshayutin)]

@chem, can we blacklist a rpm for the package upgrade, or how would you handle this?

[wes hayutin (weshayutin)]

Looks like 8-stream does have 1.18, we will be migrating victoria to stream perhaps today 3/9/2021 so the issue may just go away then

[wes hayutin (weshayutin)]

also hitting ussuri
https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c18/779458/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates-ussuri/c18e7c6/logs/undercloud/home/zuul/overcloud_update_run-Controller.log

[yatin (yatinkarel)]

<< Looks like 8-stream does have 1.18, we will be migrating victoria to stream perhaps today 3/9/2021 so the issue may just go away then
yes it's coming from c8-stream and it's likely hitting after https://review.rdoproject.org/r/#/c/32255/. And yes moving to stream will clear this.
Also that repo was added in delorean-deps due to repos change in CentOS8.3 which i think is no longer needed, so we can cleanup this. will check with other folks around it. And to resolve current issue will remove that repo from victoria and see how the job goes.

[yatin (yatinkarel)]

<< And yes moving to stream will clear this.
https://review.opendev.org/c/openstack/tripleo-quickstart/+/779315/ moving victoria to c8-stream and this should clear it for victoria.

<< And to resolve current issue will remove that repo from victoria and see how the job goes.
Removed that from ussuri and testing in https://review.opendev.org/c/openstack/tripleo-heat-templates/+/779458

[Marios Andreou (marios-b)]

seems like it is fixed by switch to stream @ https://review.opendev.org/c/openstack/tripleo-quickstart/+/779315

the victoria tripleo-ci-centos-8-scenario000-multinode-oooq-container-update has success https://zuul.openstack.org/status#779315
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_708/779315/2/check/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/70813e1/

[Marios Andreou (marios-b)]

sorry, per comment #9
actually... that is master branch... we need something in victoria to test it... so not sure yet if the switch to stream fixes the conflict apologies for confusion

[Marios Andreou (marios-b)]

posted testproject there https://review.rdoproject.org/r/#/c/32268/

[yatin (yatinkarel)]

<< Also that repo was added in delorean-deps due to repos change in CentOS8.3 which i think is no longer needed, so we can cleanup this. will check with other folks around it. And to resolve current issue will remove that repo from ussuri and see how the job goes.

Test run for ussuri is green in https://review.opendev.org/c/openstack/tripleo-heat-templates/+/779458, powertools repos being removed from delorean-deps.repo https://review.rdoproject.org/r/#/c/32269/

[wes hayutin (weshayutin)]

Thanks Yatin!