pyroute <0.6.10 -- bug with exception on removing ips

[Ian Kumlien]

There is a bug with the version included in the build that is currently done.

I can't find where and why 0.6.3 is included currently but it needs to be bumped...

As concluded in:
https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1995735

Where they did a backport of the bugfix.

[Ian Kumlien (pomac)]

Sorry, the correct url should have been this:
https://bugs.launchpad.net/ubuntu/+source/pyroute2/+bug/1995469

Even if the other one refers to it

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/kolla/+/904781

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla/+/904781
Committed: https://opendev.org/openstack/kolla/commit/63031520e51b160ba4c5429fb811dd27e10e05a4
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 63031520e51b160ba4c5429fb811dd27e10e05a4
Author: Michal Nasiadka <email address hidden>
Date: Thu Jan 4 19:08:24 2024 +0100

    [yoga-only]: Use pyroute2 from distro vendors

    pyroute2 0.6.6 derived from upper-constraints has a bug which manifests
    in Operation not permitted problems on newer kernels.

    Distro vendors (RDO, UCA and Debian OpenStack) have already backported
    the fix to their python3-pyroute2 package - let's use it.

    Closes-Bug: #2042954
    Change-Id: Iddd1f1ee6d4cd6b9a15ead323178052fe6b6e4fb

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 14.12.0 release.

[Mark Goddard (mgoddard)]

At least on Ubuntu (and possibly Debian), this fix appears not to work.

Neutron's requirements.txt in yoga specifies:

  pyroute2>=0.6.6

And upper constraints has 0.6.6. The 0.6.4 provided by Focal does not satisfy this, so pip pulls 0.6.6 from PyPI.

This can be seen in the build logs for neutron-base, e.g.

  INFO:kolla.common.utils.neutron-base:Collecting pyroute2>=0.6.6 (from neutron==20.5.1.dev24)

https://77fe6a525f1dcfeae49d-1abec0e013b8555f731bc5b1528446b7.ssl.cf2.rackcdn.com/910437/3/gate/kolla-build-ubuntu-source/09a6862/kolla/build/neutron-base.log

A workaround is to uninstall the pyroute2 command in the neutron-metadata-agent footer:

  pip uninstall pyroute2 pyroute2.core pyroute2.ethtool pyroute2.ipdb pyroute2.ipset pyroute2.ndb pyroute2.nftables pyroute2.nslink pyroute2.protocols -y

CentOS Stream 8 seems unaffected because it has python3-pyroute2 0.6.6 in the system packages.

[Mark Goddard (mgoddard)]

We haven't got to Zed yet, but will test when we get there. From what I can see, we will get pyroute2 0.7.2 due to upper-constraints.

[Maksim Malchuk (mmalchuk)]

not sure about pyroute2.protocols
https://paste.openstack.org/show/b0he8a7AvSnBox2GaAYT/

[Maksim Malchuk (mmalchuk)]

looks like this workaround don't work
https://paste.openstack.org/show/bauemnj3JS5hCkGnUTrk/

[Maksim Malchuk (mmalchuk)]

just for sure 0.6.4-3ubuntu2~cloud0 is the same version 0.6.4-3ubuntu2 with fix but from Ubuntu Cloud Archive.

[Maksim Malchuk (mmalchuk)]

update: this not reproduced on Ubuntu Focal + UCA
even previous fix not needed:
https://paste.openstack.org/show/bhRCuNgcHyKs8Gn1BDhC/

maybe this issue affects only Jammy without UCA?
anyway there is need fix issue with pyroute2.protocols

[Maksim Malchuk (mmalchuk)]

IMHO neither Focal nor Jammy not affected because kernel commit https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a6cec0bcd34264be8887791594be793b3f12719f merged in v5.18 but both Ubuntu distros uses v5.15 maximum kernel.
so https://review.opendev.org/c/openstack/kolla/+/904781 also should be reverted

[Ian Kumlien (pomac)]

So you are not allowed to diverge from the distro kernel even if you have to?

Interesting thought process...

[Maksim Malchuk (mmalchuk)]

According the documentation: https://docs.openstack.org/kolla/yoga/support_matrix.html
Yoga uses Ubuntu Focal, which can use v5.15 HWE kernel, but in transition to Zed it can be upgraded to Ubuntu Jammy with v6.5 HWE kernel and this is a case. But Zed itself not affected because uses pyroute2 v0.7.2. So this is corner case. On the IRC meeting we agreed to merge the workaround so lets wait.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla/+/913584
Committed: https://opendev.org/openstack/kolla/commit/d099063c80944961092cc4be157d31b7eee29e56
Submitter: "Zuul (22348)"
Branch: unmaintained/yoga

commit d099063c80944961092cc4be157d31b7eee29e56
Author: Michal Nasiadka <email address hidden>
Date: Mon Mar 18 17:12:22 2024 +0100

    Followup 904781 on Debian/Ubuntu

    Seems [1] fixed rpm distros, but not deb.

    [1]: https://review.opendev.org/#/q/Iddd1f1ee6d4cd6b9a15ead323178052fe6b6e4fb

    Closes-Bug: #2042954
    Change-Id: I26369f7c570c6c9d493f17509db0aa8264e6f3b2