neutron

[Functional tests] Keepalived fails to start when not existing interfaces are set in config file

Bug #1830232 reported by Slawek Kaplonski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Slawek Kaplonski

Bug Description

It looks that when not existing interfaces are given in keepalived.conf file, keepalived may not start properly.

I saw that when running functional tests from module neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase on RHEL 8 where keepalived 2.0.10 is used.
I saw in logs something like:

maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: Registering Kernel netlink reflector
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: Registering Kernel netlink command channel
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: Opening file '/tmp/tmpo_he5agd/tmpnhyku1i8/router1/keepalived.conf'.
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 7) WARNING - interface eth0 for vrrp_instance VR_1 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 17) (VR_1) tracked interface eth0 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 20) WARNING - interface eth0 for ip address 169.254.0.1/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 23) WARNING - interface eth1 for ip address 192.168.1.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 24) WARNING - interface eth2 for ip address 192.168.2.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 25) WARNING - interface eth2 for ip address 192.168.3.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 26) WARNING - interface eth10 for ip address 192.168.55.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 29) WARNING - interface eth1 for VROUTE nexthop doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 34) WARNING - interface eth4 for vrrp_instance VR_2 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 40) (VR_2) tracked interface eth4 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 43) WARNING - interface eth4 for ip address 169.254.0.2/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 46) WARNING - interface eth2 for ip address 192.168.2.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 47) WARNING - interface eth6 for ip address 192.168.3.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: (Line 48) WARNING - interface eth10 for ip address 192.168.55.0/24 doesn't exist
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: Non-existent interface specified in configuration
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived_vrrp[11267]: Stopped - used 0.000608 user time, 0.000000 system time
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived[11266]: Keepalived_vrrp exited with permanent error CONFIG. Terminating
maj 23 10:10:16 de208a364e0f82ba5124812fa88cfd47-tester-0 Keepalived[11266]: Stopped Keepalived v2.0.10 (11/12,2018)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/661042

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/661042
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=959af761cb1197bbeaed4ba1f0c3e5ef4aba3ee1
Submitter: Zuul
Branch: master

commit 959af761cb1197bbeaed4ba1f0c3e5ef4aba3ee1
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 23 17:08:56 2019 +0200

    [Functional tests] Test keepalived in namespaces

    Functional tests for keepalived should spawn processes in namespaces
    where dummy interfaces used in keepalived.conf file exists.
    Otherwise keepalived 2.0.10 (this is version used currently in RHEL 8)
    fails to start and tests are failing.

    On older versions of keepalived, like 1.3.9 used in Ubuntu 18.04,
    keepalived is logging warning about not existing interfaces but it's
    starting fine thus tests are running properly.

    So this patch adds creation of namespace for each test from
    neutron.tests.functional.agent.linux.test_keepalived module,
    creates dummy interfaces with names used in keepalived config file
    and runs keepalive process in this namespace.

    Change-Id: I54f45b8c52fc1ecce811b028f0f92e0d78d3157b
    Closes-Bug: #1830232

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/661456

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/661456
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5339544a0d37aa03bff4c1649fa77dac3a602469
Submitter: Zuul
Branch: stable/stein

commit 5339544a0d37aa03bff4c1649fa77dac3a602469
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 23 17:08:56 2019 +0200

    [Functional tests] Test keepalived in namespaces

    Functional tests for keepalived should spawn processes in namespaces
    where dummy interfaces used in keepalived.conf file exists.
    Otherwise keepalived 2.0.10 (this is version used currently in RHEL 8)
    fails to start and tests are failing.

    On older versions of keepalived, like 1.3.9 used in Ubuntu 18.04,
    keepalived is logging warning about not existing interfaces but it's
    starting fine thus tests are running properly.

    So this patch adds creation of namespace for each test from
    neutron.tests.functional.agent.linux.test_keepalived module,
    creates dummy interfaces with names used in keepalived config file
    and runs keepalive process in this namespace.

    Change-Id: I54f45b8c52fc1ecce811b028f0f92e0d78d3157b
    Closes-Bug: #1830232
    (cherry picked from commit 959af761cb1197bbeaed4ba1f0c3e5ef4aba3ee1)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 14.0.2

This issue was fixed in the openstack/neutron 14.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 15.0.0.0b1

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/684249

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/685118

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/684249
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d8eb42f1eaaa3747691cbb91db49292c66732073
Submitter: Zuul
Branch: master

commit d8eb42f1eaaa3747691cbb91db49292c66732073
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Sep 23 17:43:44 2019 +0000

    Configure keepalived interfaces according to config file

    "keepalived" process stops running if it does not find any of the
    tracked interfaces correctly configured. To prevent this in
    "KeepalivedManagerTestCase" test cases, "eth0" is set up and an IP
    address is assigned.

    Change-Id: Ibd4ef7b0db5b6830383fffcac685b9c709aae350
    Closes-Bug: #1845150
    Related-Bug: #1830232

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/train)

Reviewed: https://review.opendev.org/685118
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=eb1d0313d0dddb5e93f4d91ae28390946edab149
Submitter: Zuul
Branch: stable/train

commit eb1d0313d0dddb5e93f4d91ae28390946edab149
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Sep 23 17:43:44 2019 +0000

    Configure keepalived interfaces according to config file

    "keepalived" process stops running if it does not find any of the
    tracked interfaces correctly configured. To prevent this in
    "KeepalivedManagerTestCase" test cases, "eth0" is set up and an IP
    address is assigned.

    Change-Id: Ibd4ef7b0db5b6830383fffcac685b9c709aae350
    Closes-Bug: #1845150
    Related-Bug: #1830232
    (cherry picked from commit d8eb42f1eaaa3747691cbb91db49292c66732073)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/stein)

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/718217

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/stein)

Reviewed: https://review.opendev.org/718217
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9dc499da2bb7515ed19204ab4d46e34c7abd1ee3
Submitter: Zuul
Branch: stable/stein

commit 9dc499da2bb7515ed19204ab4d46e34c7abd1ee3
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Sep 23 17:43:44 2019 +0000

    Configure keepalived interfaces according to config file

    "keepalived" process stops running if it does not find any of the
    tracked interfaces correctly configured. To prevent this in
    "KeepalivedManagerTestCase" test cases, "eth0" is set up and an IP
    address is assigned.

    Change-Id: Ibd4ef7b0db5b6830383fffcac685b9c709aae350
    Closes-Bug: #1845150
    Related-Bug: #1830232
    (cherry picked from commit d8eb42f1eaaa3747691cbb91db49292c66732073)
    (cherry picked from commit eb1d0313d0dddb5e93f4d91ae28390946edab149)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.