Comment 14 for bug 1862703

Reverting this patch will compromise the basic functionality of remote security group again, unless we revert the patch for https://bugs.launchpad.net/neutron/+bug/1854131 as well. @yang-li Can you elaborate how this patch reopens the bug/1854131? Does that mean after moving a vm from sg1 to sg2, sg1 still somehow holds the vm's ip address in its member dictionary? If so, then I think we need a different approach to fix the issue.