Unable to update mtu on DVR snat

Bug #1933273 reported by Arnaud Morin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Slawek Kaplonski

Bug Description

When updating the mtu on a network, the new mtu is correctly set in qrouter-namespace but not in snat-namespace.

Also, a stack trace is reported in l3-agent logs stating that the qr-abcdef interface is not found.

Step by step:

- create a router in DVR mode (I also have HA, but non HA is also affected)
- add an external gw
- add a network to this router

Check the network mtu on the compute on both namespaces:
$ ip netns exec qrouter-9d5352e8-4f40-49a0-a632-9406042478b8 ip l
...
18: qr-e05ab302-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450

$ ip netns exec snat-9d5352e8-4f40-49a0-a632-9406042478b8 ip l
...
16: sg-7ab0954c-73: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450

(mtu is 1450)

- change the network mtu:
$ openstack network set --mtu 1400 MyNetwork

Check the network mtu on the compute on both namespaces:
$ ip netns exec qrouter-9d5352e8-4f40-49a0-a632-9406042478b8 ip l
...
18: qr-e05ab302-1b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400

$ ip netns exec snat-9d5352e8-4f40-49a0-a632-9406042478b8 ip l
...
16: sg-7ab0954c-73: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450

In neutron-l3-agent.log file, I can see:
ERROR neutron.agent.l3.router_info [-] Network interface qr-e05ab302-1b not found in namespace snat-9d5352e8-4f40-49a0-a632-9406042478b8.: neutron.privileged.agent.linux.ip_lib.Net[..truncated..]
ERROR neutron.agent.l3.router_info Traceback (most recent call last):
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 167, in call
ERROR neutron.agent.l3.router_info return func(*args, **kwargs)
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 1243, in process
ERROR neutron.agent.l3.router_info self._process_internal_ports()
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/l3/router_info.py", line 656, in _process_internal_ports
ERROR neutron.agent.l3.router_info interface_name, ip_cidrs, p['mtu'])
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 367, in internal_network_updated
ERROR neutron.agent.l3.router_info self._disable_ipv6_addressing_on_interface(interface_name)
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/l3/ha_router.py", line 331, in _disable_ipv6_addressing_on_interface
ERROR neutron.agent.l3.router_info ipv6_lladdr = ip_lib.get_ipv6_lladdr(device.link.address)
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 476, in address
ERROR neutron.agent.l3.router_info return self.attributes.get('link/ether')
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 509, in attributes
ERROR neutron.agent.l3.router_info self._parent.namespace)
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 247, in _wrap
ERROR neutron.agent.l3.router_info return self.channel.remote_call(name, args, kwargs)
ERROR neutron.agent.l3.router_info File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 224, in remote_call
ERROR neutron.agent.l3.router_info raise exc_type(*result[2])
ERROR neutron.agent.l3.router_info neutron.privileged.agent.linux.ip_lib.NetworkInterfaceNotFound: Network interface qr-e05ab302-1b not found in namespace snat-9d5352e8-4f40-49a0-a[..truncated..]
ERROR neutron.agent.l3.router_info

Note that I am running OpenStack Victoria

Akihiro Motoki (amotoki)
tags: added: l3-dvr-backlog
Revision history for this message
Slawek Kaplonski (slaweq) wrote :

I have env for that so I will try to reproduce and triage it at least.

Changed in neutron:
assignee: nobody → Slawek Kaplonski (slaweq)
Changed in neutron:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/799226

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/799439

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/799226
Committed: https://opendev.org/openstack/neutron/commit/b5dd6efdca0aa6e7405a55d66c7042a49ec72214
Submitter: "Zuul (22348)"
Branch: master

commit b5dd6efdca0aa6e7405a55d66c7042a49ec72214
Author: Slawek Kaplonski <email address hidden>
Date: Fri Jul 2 17:55:26 2021 +0200

    [DVR] Fix update of the MTU in the SNAT namespace

    When network's MTU is changed, Neutron sends notification about it
    to the L3 agents. In case of DVR (and DVR HA) MTU is then changed in
    the qrouter- namespace but should be also changed on snat interfaces
    in the snat namespace. And that part was missing.

    This patch adds special implementation of the internal_network_updated()
    method in the DvrEdgeRouter class so it can configure MTU also for
    in the snat namespace.

    This patch also removed passing attributes "interface_name",
    "ip_cidrs" and "mtu" to the internal_network_updated() method and adds
    "port" dict to be passed there. It is consistent with what is already
    done in e.g. internal_network_added() method and "port" dict is actually
    necessary to configure properly snat internal interface in the snat
    namespace.

    This patch adds also functional test of update network mtu for all types
    of routers as there was no such test at all.

    There is additional issue with DVR-HA which isn't fixed with that patch
    and for which follow up will be proposed. Because of that this patch is
    marked as partial fix for the related bug.

    Related-Bug: #1933273
    Change-Id: I200acfcaaae7f056ea9a563fead9ff2de8464971

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/812939

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/812950

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/812951

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/812952

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers