In patch [1] it introduced a binding of DB uniq constraint for L3
agent gateway. In some extreme case the DvrFipGatewayPortAgentBinding
is in DB while the gateway port not. The current code path only checks
the binding existence which will pass a "None" port to the following
code path that results an AttributeError.
In patch [1] it introduced a binding of DB uniq constraint for L3 rtAgentBinding
agent gateway. In some extreme case the DvrFipGatewayPo
is in DB while the gateway port not. The current code path only checks
the binding existence which will pass a "None" port to the following
code path that results an AttributeError.
[1] https:/ /review. opendev. org/#/c/ 702547/
Exception log:
2020-06-11 15:39:28.361 1285214 INFO neutron. db.l3_dvr_ db [None req-d6a41187- 2495-46bf- a424-ab7195c0ec b1 - - - - -] Floating IP Agent Gateway port for network 3fcb7702- ae0b-46b4- 807f-8ae94d656d d3 does not exist on host host-compute-1. Creating one. db.l3_dvr_ db [None req-d6a41187- 2495-46bf- a424-ab7195c0ec b1 - - - - -] Floating IP Agent Gateway port for network 3fcb7702- ae0b-46b4- 807f-8ae94d656d d3 already exists on host host-compute-1. Probably it was just created by other worker. create_ fip_agent_ gw_port_ if_not_ exists /usr/lib/ python2. 7/site- packages/ neutron/ db/l3_dvr_ db.py:927 db.l3_dvr_ db [None req-d6a41187- 2495-46bf- a424-ab7195c0ec b1 - - - - -] Floating IP Agent Gateway port None found for the destination host: host-compute-1 create_ fip_agent_ gw_port_ if_not_ exists /usr/lib/ python2. 7/site- packages/ neutron/ db/l3_dvr_ db.py:933 rpc.server [None req-d6a41187- 2495-46bf- a424-ab7195c0ec b1 - - - - -] Exception during message handling: AttributeError: 'NoneType' object has no attribute 'get' rpc.server Traceback (most recent call last): rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/server. py", line 170, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 220, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 190, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/api. py", line 91, in wrapped rpc.server setattr(e, '_RETRY_EXCEEDED', True) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 220, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 196, in force_reraise rpc.server six.reraise( self.type_ , self.value, self.tb) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/api. py", line 87, in wrapped rpc.server return f(*args, **kwargs) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_db/ api.py" , line 147, in wrapper rpc.server ectxt.value = e.inner_exc rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 220, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 196, in force_reraise rpc.server six.reraise( self.type_ , self.value, self.tb) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_db/ api.py" , line 135, in wrapper rpc.server return f(*args, **kwargs) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/api. py", line 126, in wrapped rpc.server LOG.debug("Retry wrapper got retriable exception: %s", e) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 220, in __exit__ rpc.server self.force_ reraise( ) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_utils/ excutils. py", line 196, in force_reraise rpc.server six.reraise( self.type_ , self.value, self.tb) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/api. py", line 122, in wrapped rpc.server return f(*dup_args, **dup_kwargs) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ api/rpc/ handlers/ l3_rpc. py", line 348, in get_agent_ gateway_ port rpc.server admin_ctx, network_id, host) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/l3_dvr_ db.py", line 953, in create_ fip_agent_ gw_port_ if_not_ exists rpc.server self._populate_ mtu_and_ subnets_ for_ports( context, [agent_port]) rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/l3_db. py", line 1978, in _populate_ mtu_and_ subnets_ for_ports rpc.server for p in self._each_ port_having_ fixed_ips( ports)] rpc.server File "/usr/lib/ python2. 7/site- packages/ neutron/ db/l3_db. py", line 1925, in _each_port_ having_ fixed_ips rpc.server fixed_ips = port.get( 'fixed_ ips', []) rpc.server AttributeError: 'NoneType' object has no attribute 'get' rpc.server
2020-06-11 15:39:28.370 1285214 DEBUG neutron.
2020-06-11 15:39:28.390 1285214 DEBUG neutron.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.
2020-06-11 15:39:28.391 1285214 ERROR oslo_messaging.