keepalived_use_no_track default=True breaks bionic deploys

[Corey Bryant]

[Impact]

In the latest Neutron stable release, a new config option was introduced to control using no_track in Keepalived. This config options defaults to true which is broken on releases before Focal.

The proposed fix for this in earlier releases is to change the default to False.

[Test Case]

TBD

[Regression Potential]

The regression potential is low as the fix is changing a configuration default.

--------------------------------------------------------------------------

Presumably the default of keepalived_use_no_track default=True breaks bionic deploys and the patch [1] has been backported to several bionic-based stable branches with a conflicting default config value; it's been backported back to stable/rocky.

[1] https://review.opendev.org/#/c/747862/1

This doesn't seem like a good backport to stable branches without having some logic in it that detects the version of keepalived and sets the default appropriately based on that version.

More discussion with slaweq from #openstack-neutron:

coreycb> Corey Bryant slaweq: about this backport, will it break deployments that are on bionic? https://review.opendev.org/#/c/747857/1
10:40 AM I don't think there are any stein deployments on focal
11:06 AM
<slaweq> Sławek Kapłoński coreycb: what keepalived version is available on Bionic?
11:06 AM coreycb: ahh, ok it's 1.x
11:06 AM
<coreycb> Corey Bryant slaweq: correct 1.x
11:06 AM
<slaweq> Sławek Kapłoński so it should be set to False on Bionic
11:10 AM → armax joined ⇐ priteau quit
11:12 AM
<coreycb> Corey Bryant slaweq: it would be really nice to if we could detect the version and set it appropriately

Can we revert this for all releases prior to Ussuri?

For Ubuntu, Ussuri is available by default with Focal, but we also backport Ussuri to bionic so we would have to change the default for keepalived_use_no_track default to False for bionic.

[Brian Haley (brian-haley)]

Slawek - do you want to take this?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Corey Bryant (corey.bryant)]

Rodlolfo's comment is pertient as well regarding making a run-time check similar to https://review.opendev.org/#/c/726079/1/neutron/agent/linux/dhcp.py.

https://bugs.launchpad.net/neutron/+bug/1890400/comments/1

[Alexander Gräb (alexander.graeb)]

I am voting against completely reverting the bespoken bug fix since we need this bug fix for our Ubuntu Bionic based Train deployments. We use keepalived < 2.x and set keepalived_use_no_track default=False.

I would also prefer to have a run-time check or setting the default for keepalived_use_no_track to false for older OS releases.

[Corey Bryant (corey.bryant)]

A run-time check would be ideal. Keep in mind with setting defaults statically that some releases are supported on multiple distro series. For example in ubuntu, ussuri is supported on focal and bionic. We can patch it in distro, of course, but what concerns me if we have to do that is that we can't comb through all future patches to ensure we're setting the right default. The upstream code should do the right thing depending on what version is available.

[Bernard Cafarelli (bcafarel)]

If I remember correctly, the root issue is not really https://review.opendev.org/#/c/747857 which adds a config option specifically to mitigate the issue. It was originally added uconditionally with https://review.opendev.org/#/c/721799/ (and similar backports) seen as working fine with keepalived 1.x.
When it was reported to cause issues with some versions, this configuration option was added and backported so distributions can disable it per version and build

And of course all for a runtime detection check if you can get one, I do not remember the discussions around this patch but general opinion was it was not possible or at least not easy (remember checking the version is not enough for LTS versions)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/757620

[Lajos Katona (lajos-katona)]

Bernard, Corey could you please check https://review.opendev.org/757620 , I tried to add version check, but not sure if all edge cases handled, and can't find more details of discussion around it.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/759657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/757620
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4dca7fae87d9bea01e01f786465fdb4d8231dc69
Submitter: Zuul
Branch: master

commit 4dca7fae87d9bea01e01f786465fdb4d8231dc69
Author: elajkat <email address hidden>
Date: Tue Oct 6 13:20:06 2020 +0200

    Keepalived version check

    [0] introduced keepalived_use_no_track config option which defaults to
    True.
    Here I add and extra runtime check to check keepalived version, and
    only if that version is bigger than 2.0.0 add the no_track option to
    the keepalived config file.

    [0] https://review.opendev.org/745641

    Change-Id: I16dce5b3faf62ce02535170a1b0e1ad22ee8ac30
    Closes-Bug: #1896506

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/759657
Committed: https://opendev.org/openstack/neutron/commit/2f66cb8182f6c1f07018584b7a4d721c03b5711a
Submitter: Zuul
Branch: master

commit 2f66cb8182f6c1f07018584b7a4d721c03b5711a
Author: elajkat <email address hidden>
Date: Mon Oct 26 11:07:45 2020 +0100

    Deprecate config option "keepalived_use_no_track"

    Change-Id: I448365bad076e67b32198277101f188fbfc3dece
    Related-Bug: #1896506

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/798335

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/799164

[OpenStack Infra (hudson-openstack)]

Change abandoned by "yangjianfeng <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/799164

[OpenStack Infra (hudson-openstack)]

Change abandoned by "yangjianfeng <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/798335