neutron

Non HA router - missing iptables rule for redirect metadata queries to haproxy

Bug #1947993 reported by Slawek Kaplonski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Slawek Kaplonski

Bug Description

In case of the non-HA routers (dvr and legacy) neutron-l3-agent sends notifications AFTER_CREATE and AFTER_UPDATE of the router. Metadata driver is subscribed to those notifications to prepare haproxy in the router's namespace: https://github.com/openstack/neutron/blob/8353c2adba08f9e7d5ed61589daef81aaf275fb3/neutron/agent/metadata/driver.py#L281 and https://github.com/openstack/neutron/blob/8353c2adba08f9e7d5ed61589daef81aaf275fb3/neutron/agent/metadata/driver.py#L294

The difference between those 2 functions is that in after_router_added there is called apply_metadata_nat_rules() to configure nat rules in the iptables in qrouter namespace.
In after_router_update function nat rules aren't created.

And that can cause issue when processing router in _process_added_router() will fail: https://github.com/openstack/neutron/blob/8353c2adba08f9e7d5ed61589daef81aaf275fb3/neutron/agent/l3/agent.py#L626 thus notification AFTER_CREATE router will not be called and nat rules will not be created.
Router will be processed again in next iteration by L3 agent, but this time router_info is already in the agent's router_info cache so it will be treated as updated router. Because of that haproxy will be started but NAT rules will never be created and metadata for instances will not be available.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/814892

Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814892
Committed: https://opendev.org/openstack/neutron/commit/41159bd9a4178a5f8a4fdc90b034c48fa9880cac
Submitter: "Zuul (22348)"
Branch: master

commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/817535

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/817716

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/817675

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/817676

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/817677

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/817716
Committed: https://opendev.org/openstack/neutron/commit/7486374ba30c709221ea0118ba8e5c8a438f3d28
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 7486374ba30c709221ea0118ba8e5c8a438f3d28
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Conflicts:
        neutron/agent/l3/agent.py

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d
    (cherry picked from commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac)
    (cherry picked from commit 1c17019f6c7957c95b1d12942c5898fcc111e4ab)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/817535
Committed: https://opendev.org/openstack/neutron/commit/1c17019f6c7957c95b1d12942c5898fcc111e4ab
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 1c17019f6c7957c95b1d12942c5898fcc111e4ab
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d
    (cherry picked from commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/817675
Committed: https://opendev.org/openstack/neutron/commit/7ae03f1eb62c68dff07874e28ee39af4274d3a3f
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 7ae03f1eb62c68dff07874e28ee39af4274d3a3f
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Conflicts:
        neutron/agent/l3/agent.py

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d
    (cherry picked from commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac)
    (cherry picked from commit 1c17019f6c7957c95b1d12942c5898fcc111e4ab)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/817676
Committed: https://opendev.org/openstack/neutron/commit/3732c66d7d42e11cf214036b7dd0bc644fa4ff4b
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 3732c66d7d42e11cf214036b7dd0bc644fa4ff4b
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Conflicts:
        neutron/agent/l3/agent.py

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d
    (cherry picked from commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac)
    (cherry picked from commit 1c17019f6c7957c95b1d12942c5898fcc111e4ab)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/train)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/817677
Committed: https://opendev.org/openstack/neutron/commit/faabf03a629a8cbe16161f56873b2f29f5ad7171
Submitter: "Zuul (22348)"
Branch: stable/train

commit faabf03a629a8cbe16161f56873b2f29f5ad7171
Author: Slawek Kaplonski <email address hidden>
Date: Thu Oct 21 12:06:25 2021 +0200

    Cleanup router for which processing added router failed

    In the _process_added_router() method of the L3 agent, if processing
    router will fail, router_info should be cleaned to e.g. be removed from
    the router cache so it will not be treated as updated router in next
    iteration of the agent.

    Conflicts:
        neutron/agent/l3/agent.py

    Closes-Bug: #1947993
    Change-Id: Ic0bc3d951d32efadc116708bfe518a711730429d
    (cherry picked from commit 41159bd9a4178a5f8a4fdc90b034c48fa9880cac)
    (cherry picked from commit 1c17019f6c7957c95b1d12942c5898fcc111e4ab)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.1.0

This issue was fixed in the openstack/neutron 19.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.3.0

This issue was fixed in the openstack/neutron 17.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.2.0

This issue was fixed in the openstack/neutron 18.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.0.0.0rc1

This issue was fixed in the openstack/neutron 20.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron train-eol

This issue was fixed in the openstack/neutron train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron ussuri-eol

This issue was fixed in the openstack/neutron ussuri-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.