l3ha don't set backup qg ports down

Bug #1965297 reported by Edward Hope-Morley
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
High
Unassigned
Ussuri
Fix Released
High
Unassigned
Victoria
Fix Released
High
Unassigned
Wallaby
Fix Released
High
Unassigned
Xena
Fix Released
High
Unassigned
Yoga
Fix Released
High
Unassigned
Zed
Fix Released
High
Unassigned
neutron
Fix Released
Medium
Edward Hope-Morley
neutron (Ubuntu)
Fix Released
High
Unassigned
Focal
Fix Released
High
Unassigned
Impish
Won't Fix
High
Unassigned
Jammy
Fix Released
High
Unassigned
Kinetic
Fix Released
High
Unassigned

Bug Description

The history to this request is as follows; bug 1916024 fixed an issue that subsequently had to be reverted due to a regression that it introduced (see bug 1927868) and the original issue can once again present itself in that keepalived is unable to send GARP on the qg port until the port is marked as UP by neutron which in loaded environments can sometimes take longer than keepalived will wait (e.g. when an l3-agent is restarted on a host that has hundreds of routers). The reason why qg- ports are marked as DOWN is because of the patch landed as part of bug 1859832 and as I understand it there is now consensus from upstream [1] to revert that patch as well and a better solution is needed to fix that particular issue. I have not found a bug open yet for the revert hence why I am opening this one.

[1] https://meetings.opendev.org/meetings/neutron_drivers/2022/neutron_drivers.2022-03-04-14.03.log.txt

------------

[Impact]
Please see LP bug description for full details but in short, this patch is a revert of a patch that has show instability in the field for users of Neutron L3HA.

[Test Plan]
  * Deploy Openstack with Neutron L3 HA enabled
  * Create a number of HA routers
  * Check all qrouter namespaces and ensure that the qg- port is UP in all

[Regression Potential]
Since the original patch was intended to address issues with MLDv2 it is possible that reverting it will re-introduce those issues and a new patch will need to be proposed to address that.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/834162

Changed in neutron:
status: New → In Progress
Miguel Lavalle (minsel)
Changed in neutron:
assignee: nobody → Edward Hope-Morley (hopem)
Miguel Lavalle (minsel)
Changed in neutron:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/834260

Revision history for this message
Stefan Hoffmann (mr-hopeman) wrote :

We can not simply revert bug 1859832 , as we than still face issues due to MLDv2 and other discovery related problems.
Instead, we should discuss a solution to either
rework [1], e.g. by let keeplived take the interface up and down or
find the traffic creating issues and block this, e.g. via iptables.

[1] https://bugs.launchpad.net/bugs/1859832

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by "liuyulong <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/834260
Reason: Let's go for the fix of https://review.opendev.org/c/openstack/neutron/+/839671/. Someone wants this can bring up locally.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834162
Committed: https://opendev.org/openstack/neutron/commit/36bf1df46df4de8f9ed0c19e1118480ce2e55d8a
Submitter: "Zuul (22348)"
Branch: master

commit 36bf1df46df4de8f9ed0c19e1118480ce2e55d8a
Author: Edward Hope-Morley <email address hidden>
Date: Thu Mar 17 12:56:21 2022 +0000

    Partially revert "Do not link up HA router gateway in backup node"

    This partially reverts commit c52029c39aa824a67095fbbf9e59eff769d92587.

    We revert everything except one minor addition to
    neutron/agent/l3/ha_router.py which ensures that ha_confs path is
    created when the keepalived manager is initialised.

    Closes-Bug: #1965297
    Change-Id: I14ad015c4344b32f7210c924902dac4e6ad1ae88

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/843581

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/843582

Changed in neutron (Ubuntu Impish):
status: New → Triaged
Changed in neutron (Ubuntu Jammy):
status: New → Triaged
Changed in neutron (Ubuntu Impish):
importance: Undecided → High
Changed in neutron (Ubuntu Jammy):
importance: Undecided → High
Changed in neutron (Ubuntu Kinetic):
status: New → Triaged
importance: Undecided → High
description: updated
Changed in neutron (Ubuntu Focal):
importance: Undecided → High
status: New → Triaged
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Edward, or anyone else affected,

Accepted neutron into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:20.1.0-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in neutron (Ubuntu Jammy):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-jammy
Changed in neutron (Ubuntu Impish):
status: Triaged → Fix Committed
tags: added: verification-needed-impish
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Edward, or anyone else affected,

Accepted neutron into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:19.3.0-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in cloud-archive:
status: Triaged → Fix Committed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Edward, or anyone else affected,

Accepted neutron into xena-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:xena-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-xena-needed to verification-xena-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-xena-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-xena-needed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/845406

Changed in neutron (Ubuntu Kinetic):
status: Triaged → Fix Released
Revision history for this message
Edward Hope-Morley (hopem) wrote :

i'm working on verifying the above today, apologies for the delay

Revision history for this message
Edward Hope-Morley (hopem) wrote :

jammy-proposed verified using [Test Plan] and had the following output:

# apt-cache policy neutron-common
neutron-common:
  Installed: 2:20.1.0-0ubuntu2
  Candidate: 2:20.1.0-0ubuntu2
  Version table:
 *** 2:20.1.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2:20.0.0-0ubuntu3 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu jammy/main amd64 Packages

$ juju run -a neutron-gateway -- ip netns exec qrouter-e6f9b06a-039f-48a1-b2b6-484aa61f9d8d ip link show| grep qg-6940823b-80
    3: qg-6940823b-80@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-6940823b-80@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-6940823b-80@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

tags: added: verification-done-jammy
removed: verification-needed-jammy
Revision history for this message
Edward Hope-Morley (hopem) wrote :

impish-proposed verified using [Test Plan] and had the following output:

neutron-common:
  Installed: 2:19.3.0-0ubuntu2
  Candidate: 2:19.3.0-0ubuntu2
  Version table:
 *** 2:19.3.0-0ubuntu2 500
        500 http://archive.ubuntu.com/ubuntu impish-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2:19.1.0-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu impish-updates/main amd64 Packages
     2:19.0.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu impish/main amd64 Packages

$ juju run -a neutron-gateway -- ip netns exec qrouter-e75d9cbe-fe7a-4ef4-a8d1-f676229c48d1 ip link show| grep qg-9a27c097-0e
    4: qg-9a27c097-0e@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    4: qg-9a27c097-0e@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    4: qg-9a27c097-0e@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

tags: added: verification-done-impish
removed: verification-needed-impish
Revision history for this message
Edward Hope-Morley (hopem) wrote :

xena-proposed verified using [Test Plan] and had the following output:

neutron-common:
  Installed: 2:19.3.0-0ubuntu2~cloud0
  Candidate: 2:19.3.0-0ubuntu2~cloud0
  Version table:
 *** 2:19.3.0-0ubuntu2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-proposed/xena/main amd64 Packages
        100 /var/lib/dpkg/status
     2:16.4.2-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     2:16.0.0~b3~git2020041516.5f42488a9a-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 Packages

$ juju run -a neutron-gateway -- ip netns exec qrouter-d93673ae-615d-4e3d-98ff-8dd9363e76a6 ip link show| grep qg-d0e6be73-59
    3: qg-d0e6be73-59@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    4: qg-d0e6be73-59@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-d0e6be73-59@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

tags: added: verification-done verification-xena-done
removed: verification-needed verification-xena-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package neutron - 2:20.1.0-0ubuntu2

---------------
neutron (2:20.1.0-0ubuntu2) jammy; urgency=medium

  * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
    Picked from upstream to ensure that ha_confs path is created
    when the keepalived manager is initialised (LP: #1965297).

neutron (2:20.1.0-0ubuntu1) jammy; urgency=medium

  * d/gbp.conf: Create stable/yoga branch.
  * New stable point release for OpenStack Yoga (LP: #1975632).
  * d/p/fix-setting-table-monitoring-conditions.patch: Dropped. Fixed in
    new stable point release.

 -- Corey Bryant <email address hidden> Fri, 27 May 2022 09:48:32 -0400

Changed in neutron (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for neutron has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package neutron - 2:20.1.0-0ubuntu2~cloud0
---------------

 neutron (2:20.1.0-0ubuntu2~cloud0) focal-yoga; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 neutron (2:20.1.0-0ubuntu2) jammy; urgency=medium
 .
   * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
     Picked from upstream to ensure that ha_confs path is created
     when the keepalived manager is initialised (LP: #1965297).

Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package neutron - 2:19.3.0-0ubuntu2~cloud0
---------------

 neutron (2:19.3.0-0ubuntu2~cloud0) focal-xena; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 neutron (2:19.3.0-0ubuntu2) impish; urgency=medium
 .
   * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
     Picked from upstream to ensure that ha_confs path is created
     when the keepalived manager is initialised (LP: #1965297).
 .
 neutron (2:19.3.0-0ubuntu1) impish; urgency=medium
 .
   * New stable point release for OpenStack Xena (LP: #1975634).

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/847633

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/847634

Revision history for this message
Edward Hope-Morley (hopem) wrote :

One caveat I have found with this revert is that after upgrading my l3-agent and restarting it, the qg port remains down for existing routers and this is because without this patch the agent (a) assumes that it set the port UP when it created the port and (b) if the port exists on restart it doesn't change it - all if which is expected. So, to resolve this you can either reboot your node, manually set the port to UP or you can do a --disable/--enable on the router using the openstack client. Personally i'd say manually setting the ports to UP is the quickest and least disruptive e.g.

ip netns exec snat-4fb550fd-7893-4a49-b0ec-8fc0b4dc3561 ip link set dev qg-359697da-7d up

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Uploaded to focal unapproved queue and victoria/wallaby staging PPAs.

Revision history for this message
Corey Bryant (corey.bryant) wrote : Please test proposed package

Hello Edward, or anyone else affected,

Accepted neutron into wallaby-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:wallaby-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-wallaby-needed to verification-wallaby-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-wallaby-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-wallaby-needed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Edward, or anyone else affected,

Accepted neutron into victoria-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:victoria-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-victoria-needed to verification-victoria-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-victoria-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-victoria-needed
Revision history for this message
Robie Basak (racb) wrote :

Hello Edward, or anyone else affected,

Accepted neutron into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:16.4.2-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in neutron (Ubuntu Focal):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-focal
removed: verification-done
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hello Edward, or anyone else affected,

Accepted neutron into ussuri-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ussuri-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ussuri-needed to verification-ussuri-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ussuri-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ussuri-needed
Revision history for this message
Brian Murray (brian-murray) wrote :

Ubuntu 21.10 (Impish Indri) has reached end of life, so this bug will not be fixed for that specific release.

Changed in neutron (Ubuntu Impish):
status: Fix Committed → Won't Fix
Revision history for this message
Edward Hope-Morley (hopem) wrote :

@corey.bryant seems that 2:18.4.0-0ubuntu1~cloud2 failed to build in wallaby-proposed [1]. Looks like a possible infra issue rather than a build failure. Can we trigger a new build?

[1] https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/wallaby-proposed/+build/24146430

Revision history for this message
Edward Hope-Morley (hopem) wrote :

wallaby-proposed verified using [Test Plan] and had the following output:

# apt-cache policy neutron-common
neutron-common:
  Installed: 2:18.4.0-0ubuntu1~cloud2
  Candidate: 2:18.4.0-0ubuntu1~cloud2
  Version table:
 *** 2:18.4.0-0ubuntu1~cloud2 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-proposed/wallaby/main amd64 Packages
        100 /var/lib/dpkg/status
     2:16.4.2-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     2:16.0.0~b3~git2020041516.5f42488a9a-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 Packages

juju run -a neutron-gateway -- ip netns exec qrouter-c24b722a-a530-4e1e-80ba-aa87c1a8ba34 ip link show| grep qg-73bb6b14-85
   4: qg-73bb6b14-85@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
   4: qg-73bb6b14-85@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
   4: qg-73bb6b14-85@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

tags: added: verification-wallaby-done
removed: verification-wallaby-needed
Revision history for this message
Edward Hope-Morley (hopem) wrote :

victoria-proposed verified using [Test Plan] and had the following output:

# apt-cache policy neutron-common
neutron-common:
  Installed: 2:17.4.1-0ubuntu1~cloud2
  Candidate: 2:17.4.1-0ubuntu1~cloud2
  Version table:
 *** 2:17.4.1-0ubuntu1~cloud2 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu focal-proposed/victoria/main amd64 Packages
        100 /var/lib/dpkg/status
     2:16.4.2-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     2:16.0.0~b3~git2020041516.5f42488a9a-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 Packages

$ juju run -a neutron-gateway -- ip netns exec snat-c6bd105e-a785-41cd-8385-b7abe1ac5fef ip link show| grep qg-1df6e9e5-cd
    3: qg-1df6e9e5-cd@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-1df6e9e5-cd@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-1df6e9e5-cd@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

tags: added: verification-victoria-done
removed: verification-victoria-needed
Revision history for this message
Edward Hope-Morley (hopem) wrote :

focal-updates verified using [Test Plan] and had the following output:

$ juju run -a neutron-gateway -- ip netns exec snat-e4c06d8b-7d4c-46ae-8cb2-16301d9982a1 ip link show| grep qg-be836b0e-98
    3: qg-be836b0e-98@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-be836b0e-98@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-be836b0e-98@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

# apt-cache policy neutron-common
neutron-common:
  Installed: 2:16.4.2-0ubuntu3
  Candidate: 2:16.4.2-0ubuntu3
  Version table:
 *** 2:16.4.2-0ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2:16.4.2-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     2:16.0.0~b3~git2020041516.5f42488a9a-0ubuntu2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 Packages

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Corey Bryant (corey.bryant) wrote : Update Released

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package neutron - 2:17.4.1-0ubuntu1~cloud2
---------------

 neutron (2:17.4.1-0ubuntu1~cloud2) focal-victoria; urgency=medium
 .
   * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
     Picked from upstream to ensure that ha_confs path is created
     when the keepalived manager is initialised (LP: #1965297).

tags: added: verification-doneverification-ussuri-needed
removed: verification-needed verification-ussuri-needed
tags: added: verification-done verification-ussuri-needed
removed: verification-doneverification-ussuri-needed
Revision history for this message
Edward Hope-Morley (hopem) wrote :

ussuri-proposed verified using [Test Plan] and had the following output:

$ juju run -a neutron-gateway -- ip netns exec snat-62acecc1-5813-4d66-99d3-ed6bd29afc14 ip link show| grep qg-31499712-a8
    3: qg-31499712-a8@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-31499712-a8@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    3: qg-31499712-a8@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000

# apt-cache policy neutron-common
neutron-common:
  Installed: 2:16.4.2-0ubuntu3~cloud0
  Candidate: 2:16.4.2-0ubuntu3~cloud0
  Version table:
 *** 2:16.4.2-0ubuntu3~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/ussuri/main amd64 Packages
        100 /var/lib/dpkg/status
     2:12.1.1-0ubuntu8 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     2:12.0.1-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages

tags: added: verification-ussuri-done
removed: verification-ussuri-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package neutron - 2:16.4.2-0ubuntu3

---------------
neutron (2:16.4.2-0ubuntu3) focal; urgency=medium

  * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
    Picked from upstream to ensure that ha_confs path is created
    when the keepalived manager is initialised (LP: #1965297).

 -- Corey Bryant <email address hidden> Tue, 05 Jul 2022 15:50:56 -0400

Changed in neutron (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote :

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package neutron - 2:16.4.2-0ubuntu3~cloud0
---------------

 neutron (2:16.4.2-0ubuntu3~cloud0) bionic-ussuri; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 neutron (2:16.4.2-0ubuntu3) focal; urgency=medium
 .
   * d/p/partially-revert-do-not-link-up-ha-router-gateway-in.patch:
     Picked from upstream to ensure that ha_confs path is created
     when the keepalived manager is initialised (LP: #1965297).

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/ussuri)

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/847634
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 21.0.0.0rc1

This issue was fixed in the openstack/neutron 21.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/843581
Committed: https://opendev.org/openstack/neutron/commit/7cf7ae05659b363e54fbb04d2c02f91b17763921
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 7cf7ae05659b363e54fbb04d2c02f91b17763921
Author: Edward Hope-Morley <email address hidden>
Date: Thu Mar 17 12:56:21 2022 +0000

    Partially revert "Do not link up HA router gateway in backup node"

    This partially reverts commit c52029c39aa824a67095fbbf9e59eff769d92587.

    We revert everything except one minor addition to
    neutron/agent/l3/ha_router.py which ensures that ha_confs path is
    created when the keepalived manager is initialised.

    Closes-Bug: #1965297
    Change-Id: I14ad015c4344b32f7210c924902dac4e6ad1ae88
    (cherry picked from commit 36bf1df46df4de8f9ed0c19e1118480ce2e55d8a)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/843582
Committed: https://opendev.org/openstack/neutron/commit/22f231fd8b2e145e63753712406ddbad01d0d08a
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 22f231fd8b2e145e63753712406ddbad01d0d08a
Author: Edward Hope-Morley <email address hidden>
Date: Thu Mar 17 12:56:21 2022 +0000

    Partially revert "Do not link up HA router gateway in backup node"

    This partially reverts commit c52029c39aa824a67095fbbf9e59eff769d92587.

    We revert everything except one minor addition to
    neutron/agent/l3/ha_router.py which ensures that ha_confs path is
    created when the keepalived manager is initialised.

    Closes-Bug: #1965297
    Change-Id: I14ad015c4344b32f7210c924902dac4e6ad1ae88
    (cherry picked from commit 36bf1df46df4de8f9ed0c19e1118480ce2e55d8a)
    (cherry picked from commit 7cf7ae05659b363e54fbb04d2c02f91b17763921)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.5.0

This issue was fixed in the openstack/neutron 19.5.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.3.0

This issue was fixed in the openstack/neutron 20.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/845406
Committed: https://opendev.org/openstack/neutron/commit/61e951d668c7a6da97a71b8a7d8885fe446c685e
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 61e951d668c7a6da97a71b8a7d8885fe446c685e
Author: Edward Hope-Morley <email address hidden>
Date: Thu Mar 17 12:56:21 2022 +0000

    Partially revert "Do not link up HA router gateway in backup node"

    This partially reverts commit c52029c39aa824a67095fbbf9e59eff769d92587.

    We revert everything except one minor addition to
    neutron/agent/l3/ha_router.py which ensures that ha_confs path is
    created when the keepalived manager is initialised.

    Closes-Bug: #1965297
    Change-Id: I14ad015c4344b32f7210c924902dac4e6ad1ae88
    (cherry picked from commit 36bf1df46df4de8f9ed0c19e1118480ce2e55d8a)
    (cherry picked from commit 7cf7ae05659b363e54fbb04d2c02f91b17763921)
    (cherry picked from commit 22f231fd8b2e145e63753712406ddbad01d0d08a)

tags: added: in-stable-wallaby
tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/847633
Committed: https://opendev.org/openstack/neutron/commit/6fd2dd1aa37ad3dd44b76be063139e2fcf8ccf5d
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 6fd2dd1aa37ad3dd44b76be063139e2fcf8ccf5d
Author: Edward Hope-Morley <email address hidden>
Date: Thu Mar 17 12:56:21 2022 +0000

    Partially revert "Do not link up HA router gateway in backup node"

    This partially reverts commit c52029c39aa824a67095fbbf9e59eff769d92587.

    We revert everything except one minor addition to
    neutron/agent/l3/ha_router.py which ensures that ha_confs path is
    created when the keepalived manager is initialised.

    Closes-Bug: #1965297
    Change-Id: I14ad015c4344b32f7210c924902dac4e6ad1ae88
    (cherry picked from commit 36bf1df46df4de8f9ed0c19e1118480ce2e55d8a)
    (cherry picked from commit 7cf7ae05659b363e54fbb04d2c02f91b17763921)
    (cherry picked from commit 22f231fd8b2e145e63753712406ddbad01d0d08a)
    (cherry picked from commit 61e951d668c7a6da97a71b8a7d8885fe446c685e)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron victoria-eom

This issue was fixed in the openstack/neutron victoria-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron wallaby-eom

This issue was fixed in the openstack/neutron wallaby-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.