neutron.plugins.ml2.drivers.agent._common_agent KeyError: 'gateway' . ERROR

[hypargus]

Hi Everyone , I installed train on Ubuntu 18.0.4 Server . I've a problem with neutron . when I run openstack create subnet , my subnet created , but when I check neutron-linuxbridge-agent.log in controller node , it return this error . I configured neutron as provider

ERROR neutron.plugins.ml2.drivers.agent._common_agent KeyError: 'gateway' . ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 402, in _update_interface_ip_details .

ml2.conf
[DEFAULT]
core_plugin = ml2
service_plugins =
allow_overlapping_ips = true
transport_url = rabbit://openstack:XXXX_Rabbit@controller
auth_strategy = keystone
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
[agent]
root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"
[cors]
[database]
connection = mysql+pymysql://neutron:NEUTRON_XXXX@controller/neutron
[ironic]
[keystone_authtoken]
www_authenticate_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = NEUTRON@XXXX
[nova]
auth_url = http://controller:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password = XXXX@Nova
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = XXXX_Rabbit
[oslo_middleware]
[oslo_policy]
[privsep]
[quotas]
[ssl]

l3_agent.ini

[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:enp6s0
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = false

[YAMAMOTO Takashi (yamamoto)]

can you provide the entire linuxbridge-agent log, preferably with DEBUG level?

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

I think this is my fault. In I5e5e9f6981024317773979d9d2d77db3f5e7ec98 I changed the dict parameters returned in "list_ip_routes" (and thus "IpRouteCommand.get_gateway") to match input and output parameters. Instead of using "via" as input parameter and "gateway" as output, I used "via" for both. But I forgot to change this one.

Regards.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/698287

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698287
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7
Submitter: Zuul
Branch: master

commit b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Dec 10 15:50:20 2019 +0000

    Use "via" in gateway dictionary in Linux Bridge agent

    The gateway IP address in the gateway dictionary returned by
    "ip_lib.list_ip_routes" is stored in "via".

    "priority" parameter is changed to "metric", to match input and
    output parameters.

    Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
    Closes-Bug: #1855759

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/699937

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699937
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=124680084c6f921b49df5da0095ff80053ca0e52
Submitter: Zuul
Branch: stable/train

commit 124680084c6f921b49df5da0095ff80053ca0e52
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Dec 10 15:50:20 2019 +0000

    Use "via" in gateway dictionary in Linux Bridge agent

    The gateway IP address in the gateway dictionary returned by
    "ip_lib.list_ip_routes" is stored in "via".

    "priority" parameter is changed to "metric", to match input and
    output parameters.

    Change-Id: I67ae473dca8d706f963c3b55b9410f9a79d7f32b
    Closes-Bug: #1855759
    (cherry picked from commit b99765df8f1d1d6d3ceee3d481d1e6ee1b2200e7)

[Tristan Sullivan (tristansullivan)]

Hi,

I had the issue described here, because the version of openstack-train provided by Centos7 predates the fix. I applied the fix, but then ran into another problem. The linuxbridge-agent log file started filling up with a different error message. Here's the end of the stack trace:

2020-02-14 10:44:36.529 100143 ERROR neutron.plugins.ml2.drivers.agent._common_agent
metric = gateway['metric'] - 1
2020-02-14 10:44:36.529 100143 ERROR neutron.plugins.ml2.drivers.agent._common_agent TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

gateway['metric'] comes from line 1503 of neutron/agent/linux/ip_lib.py, in list_ip_routes:

'metric': get_attr(route, 'RTA_PRIORITY')

I have routes with metric 0. In this case, it seems that get_attr returns None instead of 0. Then, in line 400 of neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py, it tries to do

metric = gateway['metric'] - 1,

and gives the error indicated in the log file. As a workaround I added

if value['metric'] == None:
            value['metric'] = 1

to ip_lib.py in list_ip_routes, which makes it work, but is probably not the best solution. I'm new to Openstack, so I wasn't sure if I should make a new bug report or put this here. If I should make a new bug report instead, or if there is other information I should provide, please let me know, and I will do so.

Thank you,

Tristan

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hi Tristan:

I have confirmed that pyroute2 returns "None" if the metric is not explicitly set.

The default route metric values should be set according to the default values defined in [1].

Regards.

[1] http://www.man7.org/linux/man-pages/man8/route.8.html

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/708086

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.2 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708086
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7593f95a741aec62d9956175ce6d3ce1b1abb9a0
Submitter: Zuul
Branch: master

commit 7593f95a741aec62d9956175ce6d3ce1b1abb9a0
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Feb 17 10:28:12 2020 +0000

    Set a default IP route metric in ip_lib.list_ip_routes

    By default, if no metric is defined, the kernel interprets the
    highest value (0).

    The current implementation, using pyroute2, is a translation from
    the CLI command "ip route". This command uses the netlink API to
    communicate with the kernel. In IPv6, when the metric value is not
    set is translated as 1024 as default [1].

    [1]https://access.redhat.com/solutions/3659171

    Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
    Related-Bug: #1855759

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/722384

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/722384
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f900561550966b0d76936a292f658e16a97c1c00
Submitter: Zuul
Branch: stable/train

commit f900561550966b0d76936a292f658e16a97c1c00
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Feb 17 10:28:12 2020 +0000

    Set a default IP route metric in ip_lib.list_ip_routes

    By default, if no metric is defined, the kernel interprets the
    highest value (0).

    The current implementation, using pyroute2, is a translation from
    the CLI command "ip route". This command uses the netlink API to
    communicate with the kernel. In IPv6, when the metric value is not
    set is translated as 1024 as default [1].

    [1]https://access.redhat.com/solutions/3659171

    Change-Id: I0c5f9e320bbbf314a2d6a22c515bf903de84cdaf
    Related-Bug: #1855759
    (cherry picked from commit 7593f95a741aec62d9956175ce6d3ce1b1abb9a0)