allowed-address-pairs only update ipset on one compute node

[yujie]

1. Two vms run on the same network but different compute nodes.
     vm1(100.100.100.3) on CN1
     vm2(100.100.100.4) on CN2
2. both vms bind to securitygroup sg1, sg1 has two rules:
     a) egress: all protocol, 0.0.0.0/0
     b) ingress: all protocol, remote sg: sg1
3. vm1 and vm2 could ping each other successfully as we expect.
4. update port belong to vm1 by using: neutron port-update 4d436802-fa9f-4552-97ee-7626f691b8ca --allowed-address-pairs type=dict list=true ip_address=100.100.100.10
5. change IP of vm1 to 100.100.100.10. Now vm2 could ping vm1 successfully, but vm1 could not ping vm2.

Then check the ipset on CN1: ipset list
Name: NETIPv4f766bf09-a5fa-4901-9
Type: hash:net
Revision: 3
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16880
References: 1
Members:
100.100.100.3
100.100.100.10
100.100.100.4

Check ipset on CN2: ipset list
Name: NETIPv4f766bf09-a5fa-4901-9
Type: hash:net
Revision: 3
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 16848
References: 1
Members:
100.100.100.4
100.100.100.3

If add the IP (100.100.100.10) to IPSET NETIPv4f766bf09-a5fa-4901-9 on CN2 , vm1 could ping vm2 successfully.

I use kilo release, not sure master have this problem.

[Ryan Moats (rmoats)]

Yujie, can you please check if master has this problem?

[yujie (16189455-d)]

Hi Ryan, this test need two compute node. But for master release I only have devstack environment which is not satisfied.

[yujie (16189455-d)]

Could any body give a test in master?

[Tore Anderson (toreanderson)]

I can confirm this bug. Observed it on Mitaka using IPv6 addressing. Out of three compute nodes that has the ipset in question, only one was updated (the one where the port in question exists on).

Tore

[Perry (panxia6679)]

we met the problem in mitaka without explicitly using allowed-address-pairs. As result, one of member is missed from ipset group. All other symptom looks the same.

[Kevin Benton (kevinbenton)]

Bumping to high and unassigned so we can get someone to look at this.

[Hunt Xu (huntxu)]

This issue no long affects master.

My test environment is devstack with two compute nodes, neutron version is master commit 38434dfbc14a1d5d2f4a7985b31f0cf5b0a0ccdb (Oct 26, 2017).

After updating one port's allowed address pairs, I can see ipset updated on both compute nodes. And vms can ping each other.

Looking at the codes, this issue is fixed by commit b9b598040da1b8d3e860b39d9027c10f8576ee33([1], [2]). With this commit, update_port[3] will always notify notify_sg_on_port_change[4], which will later ensure SG members are updated on all the compute nodes with allowed_address_pairs changes being taken into account.

[1]. https://bugs.launchpad.net/neutron/+bug/1666424
[2]. https://git.openstack.org/cgit/openstack/neutron/commit/?id=b9b598040da1b8d3e860b39d9027c10f8576ee33
[3]. https://github.com/openstack/neutron/blob/d8121eab3568fae68b8590178ade7ccbf295316d/neutron/plugins/ml2/plugin.py#L1391
[4]. https://github.com/openstack/neutron/blob/d8121eab3568fae68b8590178ade7ccbf295316d/neutron/db/securitygroups_rpc_base.py#L45-L51