l3-agent network_update() can throw an excetion if router ex_gw_port is None

Bug #1724043 reported by Brian Haley
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Brian Haley

Bug Description

I've seen this error a couple of times in downstream testing, but looks like it could be just as broken upstream:

2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server [req-a1152197-d8b1-4e34-ae63-8a94fd69ebcd faf66db1b6de4c56a9925b9e5aa3369d c977198dffa24e6f8e9e8c8c4cf3211c - - -] Exception during message handling: TypeError: 'NoneType' object has no attribute '__getitem__'
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in network_update
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server if any(port_belongs(p) for p in ports):
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in <genexpr>
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server if any(port_belongs(p) for p in ports):
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 445, in <lambda>
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server port_belongs = lambda p: p['network_id'] == network_id
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server TypeError: 'NoneType' object has no attribute '__getitem__'
2017-10-09 18:36:29.507 126916 ERROR oslo_messaging.rpc.server

Since ri.ex_gw_port can be None, that can cause an exception when looking for ports we might have in that network. Here's an example if the port is None:

  ports = itertools.chain(ri.internal_ports, [ri.ex_gw_port])

>>> import itertools
>>> foo = itertools.chain([], [None])
>>> port_belongs = lambda p: p['network_id'] == network_id
>>> any(port_belongs(p) for p in foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <genexpr>
  File "<stdin>", line 1, in <lambda>
TypeError: 'NoneType' object has no attribute '__getitem__'

We need to check if it's None and use [] instead.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/512382

Changed in neutron:
status: New → In Progress
tags: added: pike-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/514578

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/512382
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2cea213d94e3ac7675d62491cec6c85253285b92
Submitter: Zuul
Branch: master

commit 2cea213d94e3ac7675d62491cec6c85253285b92
Author: Brian Haley <email address hidden>
Date: Mon Oct 16 14:35:25 2017 -0400

    Do not try and iterate [None] in l3-agent network_update()

    Since ri.ex_gw_port can be None, the l3-agent can throw an
    exception when looking for ports it might have in a given
    network.

    Change-Id: I3ab3e9c012022cd7eefa5c609ca9540649079ad3
    Closes-bug: #1724043

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/pike)

Reviewed: https://review.openstack.org/514578
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9414425ffa9210da846af841ac0493a48deddc0f
Submitter: Zuul
Branch: stable/pike

commit 9414425ffa9210da846af841ac0493a48deddc0f
Author: Brian Haley <email address hidden>
Date: Mon Oct 16 14:35:25 2017 -0400

    Do not try and iterate [None] in l3-agent network_update()

    Since ri.ex_gw_port can be None, the l3-agent can throw an
    exception when looking for ports it might have in a given
    network.

    (cherry picked from 2cea213d94e3ac7675d62491cec6c85253285b92)

    Change-Id: I3ab3e9c012022cd7eefa5c609ca9540649079ad3
    Closes-bug: #1724043

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 11.0.2

This issue was fixed in the openstack/neutron 11.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.0.0.0b2

This issue was fixed in the openstack/neutron 12.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.