Fix race condition with enabling SG on many ports at once
When there are many calls to enable security groups on ports there
can be sometimes race condition between refresh recource_cache
with data get by "pull" call to neutron server and data received
with "push" rpc message from neutron server.
In such case when "push" message comes with information about
updated port (with enabled port_security), in local cache this port
is already updated so local AFTER_UPDATE call is not called for
such port and its rules in firewall are not updated.
It happend quite often in fullstack security groups test because
there are 4 ports created in this test and all 4 are updated to
apply SG to it one by one.
And here's what happen then in details:
1. port 1 was updated in neutron-server so it sends push notification
to L2 agent to update security groups,
2. port 1 info was saved in resource cache on L2 agent's side and agent
started to configure security groups for this port,
3. as one of steps L2 agent called SecurityGroupServerAPIShim._select_ips_for_remote_group() method;
In that method RemoteResourceCache.get_resources() is called and this
method asks neutron-server for details about ports from given security_group,
4. in the meantime neutron-server got port update call for second port
(with same security group) so it sends to L2 agent informations about 2
ports (as a reply to request sent from L2 agent in step 3),
5. resource cache updates informations about two ports in local cache,
returns its data to SecurityGroupServerAPIShim._select_ips_for_remote_group() and all
looks fine,
6. but now L2 agent receives push notification with info that port 2 is
updated (changed security groups), so it checks info about this port
in local cache,
7. in local cache info about port 2 is already WITH updated security
group so RemoteResourceCache doesn't trigger local notification about
port AFTER UPDATE and L2 agent doesn't know that security groups for this
port should be changed
This patch fixes it by changing way how items are updated in
the resource_cache.
For now it is done with record_resource_update() method instead of
writing new values directly to resource_cache._type_cache dict.
Due to that if resource will be updated during "pull" call to neutron
server, local AFTER_UPDATE will still be triggered for such resource.
Reviewed: https:/ /review. openstack. org/536367 /git.openstack. org/cgit/ openstack/ neutron/ commit/ ?id=725df3e0382 e048391fac109ea 57920683eaf4d0
Committed: https:/
Submitter: Zuul
Branch: master
commit 725df3e0382e048 391fac109ea5792 0683eaf4d0
Author: Sławek Kapłoński <email address hidden>
Date: Mon Jan 22 14:01:30 2018 +0100
Fix race condition with enabling SG on many ports at once
When there are many calls to enable security groups on ports there
can be sometimes race condition between refresh recource_cache
with data get by "pull" call to neutron server and data received
with "push" rpc message from neutron server.
In such case when "push" message comes with information about
updated port (with enabled port_security), in local cache this port
is already updated so local AFTER_UPDATE call is not called for
such port and its rules in firewall are not updated.
It happend quite often in fullstack security groups test because
SecurityGroupSe rverAPIShim. _select_ ips_for_ remote_ group() method; ache.get_ resources( ) is called and this
security_ group,
SecurityGroupSe rverAPIShim. _select_ ips_for_ remote_ group() and all
there are 4 ports created in this test and all 4 are updated to
apply SG to it one by one.
And here's what happen then in details:
1. port 1 was updated in neutron-server so it sends push notification
to L2 agent to update security groups,
2. port 1 info was saved in resource cache on L2 agent's side and agent
started to configure security groups for this port,
3. as one of steps L2 agent called
In that method RemoteResourceC
method asks neutron-server for details about ports from given
4. in the meantime neutron-server got port update call for second port
(with same security group) so it sends to L2 agent informations about 2
ports (as a reply to request sent from L2 agent in step 3),
5. resource cache updates informations about two ports in local cache,
returns its data to
looks fine,
6. but now L2 agent receives push notification with info that port 2 is
updated (changed security groups), so it checks info about this port
in local cache,
7. in local cache info about port 2 is already WITH updated security
group so RemoteResourceCache doesn't trigger local notification about
port AFTER UPDATE and L2 agent doesn't know that security groups for this
port should be changed
This patch fixes it by changing way how items are updated in resource_ update( ) method instead of cache._ type_cache dict.
the resource_cache.
For now it is done with record_
writing new values directly to resource_
Due to that if resource will be updated during "pull" call to neutron
server, local AFTER_UPDATE will still be triggered for such resource.
Change-Id: I5a62cc5731c5ba 571506a3aa26303 a1b0290d37b
Closes-Bug: #1742401