neutron.plugins.ml2.drivers.agent._common_agent KeyError: 'gateway' . ERROR

Bug #1855759 reported by hypargus on 2019-12-09
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
High
Rodolfo Alonso

Bug Description

Hi Everyone , I installed train on Ubuntu 18.0.4 Server . I've a problem with neutron . when I run openstack create subnet , my subnet created , but when I check neutron-linuxbridge-agent.log in controller node , it return this error . I configured neutron as provider

ERROR neutron.plugins.ml2.drivers.agent._common_agent KeyError: 'gateway' . ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 402, in _update_interface_ip_details .

ml2.conf
[DEFAULT]
core_plugin = ml2
service_plugins =
allow_overlapping_ips = true
transport_url = rabbit://openstack:XXXX_Rabbit@controller
auth_strategy = keystone
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
[agent]
root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"
[cors]
[database]
connection = mysql+pymysql://neutron:NEUTRON_XXXX@controller/neutron
[ironic]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = NEUTRON@XXXX
[nova]
auth_url = http://controller:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password = XXXX@Nova
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = XXXX_Rabbit
[oslo_middleware]
[oslo_policy]
[privsep]
[quotas]
[ssl]

l3_agent.ini

[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:enp6s0
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = false

hypargus (hypargus) on 2019-12-09
tags: added: neutron
YAMAMOTO Takashi (yamamoto) wrote :

can you provide the entire linuxbridge-agent log, preferably with DEBUG level?

Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)

Hello:

I think this is my fault. In I5e5e9f6981024317773979d9d2d77db3f5e7ec98 I changed the dict parameters returned in "list_ip_routes" (and thus "IpRouteCommand.get_gateway") to match input and output parameters. Instead of using "via" as input parameter and "gateway" as output, I used "via" for both. But I forgot to change this one.

Regards.

Fix proposed to branch: master
Review: https://review.opendev.org/698287

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → High

Reviewed: https://review.opendev.org/698287
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7
Submitter: Zuul
Branch: master

commit b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Dec 10 15:50:20 2019 +0000

    Use "via" in gateway dictionary in Linux Bridge agent

    The gateway IP address in the gateway dictionary returned by
    "ip_lib.list_ip_routes" is stored in "via".

    "priority" parameter is changed to "metric", to match input and
    output parameters.

    Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
    Closes-Bug: #1855759

Changed in neutron:
status: In Progress → Fix Released

Reviewed: https://review.opendev.org/699937
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=124680084c6f921b49df5da0095ff80053ca0e52
Submitter: Zuul
Branch: stable/train

commit 124680084c6f921b49df5da0095ff80053ca0e52
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Dec 10 15:50:20 2019 +0000

    Use "via" in gateway dictionary in Linux Bridge agent

    The gateway IP address in the gateway dictionary returned by
    "ip_lib.list_ip_routes" is stored in "via".

    "priority" parameter is changed to "metric", to match input and
    output parameters.

    Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
    Closes-Bug: #1855759
    (cherry picked from commit b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7)

tags: added: in-stable-train

Hi,

I had the issue described here, because the version of openstack-train provided by Centos7 predates the fix. I applied the fix, but then ran into another problem. The linuxbridge-agent log file started filling up with a different error message. Here's the end of the stack trace:

2020-02-14 10:44:36.529 100143 ERROR neutron.plugins.ml2.drivers.agent._common_agent
metric = gateway['metric'] - 1
2020-02-14 10:44:36.529 100143 ERROR neutron.plugins.ml2.drivers.agent._common_agent TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

gateway['metric'] comes from line 1503 of neutron/agent/linux/ip_lib.py, in list_ip_routes:

'metric': get_attr(route, 'RTA_PRIORITY')

I have routes with metric 0. In this case, it seems that get_attr returns None instead of 0. Then, in line 400 of neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py, it tries to do

metric = gateway['metric'] - 1,

and gives the error indicated in the log file. As a workaround I added

if value['metric'] == None:
            value['metric'] = 1

to ip_lib.py in list_ip_routes, which makes it work, but is probably not the best solution. I'm new to Openstack, so I wasn't sure if I should make a new bug report or put this here. If I should make a new bug report instead, or if there is other information I should provide, please let me know, and I will do so.

Thank you,

Tristan

Hi Tristan:

I have confirmed that pyroute2 returns "None" if the metric is not explicitly set.

The default route metric values should be set according to the default values defined in [1].

Regards.

[1] http://www.man7.org/linux/man-pages/man8/route.8.html

This issue was fixed in the openstack/neutron 15.0.2 release.

Reviewed: https://review.opendev.org/708086
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7593f95a741aec62d9956175ce6d3ce1b1abb9a0
Submitter: Zuul
Branch: master

commit 7593f95a741aec62d9956175ce6d3ce1b1abb9a0
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Feb 17 10:28:12 2020 +0000

    Set a default IP route metric in ip_lib.list_ip_routes

    By default, if no metric is defined, the kernel interprets the
    highest value (0).

    The current implementation, using pyroute2, is a translation from
    the CLI command "ip route". This command uses the netlink API to
    communicate with the kernel. In IPv6, when the metric value is not
    set is translated as 1024 as default [1].

    [1]https://access.redhat.com/solutions/3659171

    Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
    Related-Bug: #1855759

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers