keepalived_use_no_track default=True breaks bionic deploys

Bug #1896506 reported by Corey Bryant on 2020-09-21
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Undecided
Unassigned
Stein
Undecided
Unassigned
Train
Undecided
Unassigned
Ussuri
Undecided
Unassigned
neutron
High
Lajos Katona
neutron (Ubuntu)
Undecided
Unassigned

Bug Description

[Impact]

In the latest Neutron stable release, a new config option was introduced to control using no_track in Keepalived. This config options defaults to true which is broken on releases before Focal.

The proposed fix for this in earlier releases is to change the default to False.

[Test Case]

TBD

[Regression Potential]

The regression potential is low as the fix is changing a configuration default.

--------------------------------------------------------------------------

Presumably the default of keepalived_use_no_track default=True breaks bionic deploys and the patch [1] has been backported to several bionic-based stable branches with a conflicting default config value; it's been backported back to stable/rocky.

[1] https://review.opendev.org/#/c/747862/1

This doesn't seem like a good backport to stable branches without having some logic in it that detects the version of keepalived and sets the default appropriately based on that version.

More discussion with slaweq from #openstack-neutron:

coreycb> Corey Bryant slaweq: about this backport, will it break deployments that are on bionic? https://review.opendev.org/#/c/747857/1
10:40 AM I don't think there are any stein deployments on focal
11:06 AM
<slaweq> Sławek Kapłoński coreycb: what keepalived version is available on Bionic?
11:06 AM coreycb: ahh, ok it's 1.x
11:06 AM
<coreycb> Corey Bryant slaweq: correct 1.x
11:06 AM
<slaweq> Sławek Kapłoński so it should be set to False on Bionic
11:10 AM → armax joined ⇐ priteau quit
11:12 AM
<coreycb> Corey Bryant slaweq: it would be really nice to if we could detect the version and set it appropriately

Can we revert this for all releases prior to Ussuri?

For Ubuntu, Ussuri is available by default with Focal, but we also backport Ussuri to bionic so we would have to change the default for keepalived_use_no_track default to False for bionic.

description: updated
description: updated
summary: - keepalived_us_no_track default=True breaks bionic deploys
+ keepalived_use_no_track default=True breaks bionic deploys
description: updated
Brian Haley (brian-haley) wrote :

Slawek - do you want to take this?

Changed in neutron:
importance: Undecided → High
status: New → Confirmed
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in neutron (Ubuntu):
status: New → Confirmed
Corey Bryant (corey.bryant) wrote :

Rodlolfo's comment is pertient as well regarding making a run-time check similar to https://review.opendev.org/#/c/726079/1/neutron/agent/linux/dhcp.py.

https://bugs.launchpad.net/neutron/+bug/1890400/comments/1

I am voting against completely reverting the bespoken bug fix since we need this bug fix for our Ubuntu Bionic based Train deployments. We use keepalived < 2.x and set keepalived_use_no_track default=False.

I would also prefer to have a run-time check or setting the default for keepalived_use_no_track to false for older OS releases.

Corey Bryant (corey.bryant) wrote :

A run-time check would be ideal. Keep in mind with setting defaults statically that some releases are supported on multiple distro series. For example in ubuntu, ussuri is supported on focal and bionic. We can patch it in distro, of course, but what concerns me if we have to do that is that we can't comb through all future patches to ensure we're setting the right default. The upstream code should do the right thing depending on what version is available.

tags: added: l3-ha
Bernard Cafarelli (bcafarel) wrote :

If I remember correctly, the root issue is not really https://review.opendev.org/#/c/747857 which adds a config option specifically to mitigate the issue. It was originally added uconditionally with https://review.opendev.org/#/c/721799/ (and similar backports) seen as working fine with keepalived 1.x.
When it was reported to cause issues with some versions, this configuration option was added and backported so distributions can disable it per version and build

And of course all for a runtime detection check if you can get one, I do not remember the discussions around this patch but general opinion was it was not possible or at least not easy (remember checking the version is not enough for LTS versions)

Fix proposed to branch: master
Review: https://review.opendev.org/757620

Changed in neutron:
assignee: nobody → Lajos Katona (lajos-katona)
status: Confirmed → In Progress
Lajos Katona (lajos-katona) wrote :

Bernard, Corey could you please check https://review.opendev.org/757620 , I tried to add version check, but not sure if all edge cases handled, and can't find more details of discussion around it.

Changed in cloud-archive:
status: New → Invalid
description: updated

Reviewed: https://review.opendev.org/757620
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4dca7fae87d9bea01e01f786465fdb4d8231dc69
Submitter: Zuul
Branch: master

commit 4dca7fae87d9bea01e01f786465fdb4d8231dc69
Author: elajkat <email address hidden>
Date: Tue Oct 6 13:20:06 2020 +0200

    Keepalived version check

    [0] introduced keepalived_use_no_track config option which defaults to
    True.
    Here I add and extra runtime check to check keepalived version, and
    only if that version is bigger than 2.0.0 add the no_track option to
    the keepalived config file.

    [0] https://review.opendev.org/745641

    Change-Id: I16dce5b3faf62ce02535170a1b0e1ad22ee8ac30
    Closes-Bug: #1896506

Changed in neutron:
status: In Progress → Fix Released

Reviewed: https://review.opendev.org/759657
Committed: https://opendev.org/openstack/neutron/commit/2f66cb8182f6c1f07018584b7a4d721c03b5711a
Submitter: Zuul
Branch: master

commit 2f66cb8182f6c1f07018584b7a4d721c03b5711a
Author: elajkat <email address hidden>
Date: Mon Oct 26 11:07:45 2020 +0100

    Deprecate config option "keepalived_use_no_track"

    Change-Id: I448365bad076e67b32198277101f188fbfc3dece
    Related-Bug: #1896506

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers