Ipset Race Condition

[Rene Soto]

Detailed bug description:
It was determined that the fix associated with the following upstream Neutron bug (with release fixed in Juno and Kilo) is not present in MOS 6.1 and MOS 7.0:
https://bugs.launchpad.net/neutron/+bug/1491131

Steps to reproduce:
According to the upstream Neutron LP bug, this is difficult to reproduce, but this is sometimes reproduced by creating a security group rule that has a rule which is a source group back to itself, then deleting multiple instances that with that source group.

Expected results:
-to catch when ipset tries to remove an ipset that has already been removed

Actual result:
-OVS agent churns forever trying to delete an ipset that doesn't exist
-Iptables attempts to apply rules for an ipset that was not added
-Ipset churns trying to remove ips

Reproducibility: Yes
Workaround: Restart neutron OVS agent on compute node

Impact:
Newly scheduled instances not obtaining IPs

Description of the environment:
- Operation system: CentOS 6.5
- Versions of components: MOS 6.1, 7.0
- Reference architecture: -
- Network model: Neutron + OVS (with GRE)
- Related projects installed: N/A
Additional information: N/A

[Dina Belova (dbelova)]

Marking as confirmed and moving to 7.0 and 6.1 updates. MOS-maintenance team, please take a look on the u[stream fix and if it can be applied to earlier Neutron versions.

[Alexey Stupnikov (astupnikov)]

Steps to reproduce from upstream bug:
  - create a security group that has a rule which is a source group back to itself (default group has such kind of rules);
  - launch and delete multiple instances with that source group.

[Alexey Stupnikov (astupnikov)]

I have used attached script to reproduce this bug, but has no success.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: shihanzhang <email address hidden>
Review: https://review.fuel-infra.org/22991

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Change author: shihanzhang <email address hidden>
Review: https://review.fuel-infra.org/22995

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-6.1/2014.2
Change author: shihanzhang <email address hidden>
Review: https://review.fuel-infra.org/22996

[Alexey Stupnikov (astupnikov)]

According to our police, we have to ask neutron team to confirm acceptance of proposed patches to stable branches.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexey Stupnikov <email address hidden> on branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/22995
Reason: Can't reproduce on MOS6.1

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/22991
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: fe4733bdad23502b35a3b7bcaa2ddad64689662a
Author: shihanzhang <email address hidden>
Date: Wed Jul 6 13:07:33 2016

Fix ipset can't be destroyed when last rule is deleted

when it deletes a security group all rules, it should
include this sg information in RPC method
'security_group_info_for_devices', otherwise the ports
in this sg can't corrcectly update their iptables and
ipset sets.

Change-Id: Ibb071ce84590bd46cda2c1e010a566e75e22b4d2
Closes-bug: #1596682
(cherry picked from commit 764f018f50ac7cd42c29efeabaccbb5aec21f6f4)

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/22996
Submitter: Vitaly Sedelnik <email address hidden>
Branch: openstack-ci/fuel-6.1/2014.2

Commit: 8bbad206564db4555a8419fec149b5a7a06d1897
Author: shihanzhang <email address hidden>
Date: Thu Jul 7 16:29:33 2016

Fix ipset can't be destroyed when last rule is deleted

when it deletes a security group all rules, it should
include this sg information in RPC method
'security_group_info_for_devices', otherwise the ports
in this sg can't corrcectly update their iptables and
ipset sets.

Change-Id: Ibb071ce84590bd46cda2c1e010a566e75e22b4d2
Closes-bug: #1596682
(cherry picked from commit 764f018f50ac7cd42c29efeabaccbb5aec21f6f4)

[Alexey Stupnikov (astupnikov)]

We have merged the patches since they were merged to upstream and our cores confirmed their importance for MOS product.

[Alexey Stupnikov (astupnikov)]

There is no need to verify those fixes, so they can be moved to Fix Released state automatically.

[TatyanaGladysheva (tgladysheva)]

Fix is available in 7.0 MU5 updates. Move to Fix Released.