Port create/delete issue in etcd when setting/removing Neutron Router gateway

[Yichen Wang]

Steps to reproduce on a HA setup on OpenStack Newton.

(1) Create a Neutron Router, and set up an external gateway with it. You will see below entries are created in etcd:
/networking-vpp/nodes/c45-control-1/ports
/networking-vpp/nodes/c45-control-1/ports/084973a8-3990-4a84-b13c-adbcf8ca4640
/networking-vpp/nodes/c45-control-1/ports/6409b006-b256-4353-abd9-bd2582488def
/networking-vpp/nodes/c45-control-2
/networking-vpp/nodes/c45-control-2/ports
/networking-vpp/nodes/c45-control-2/ports/084973a8-3990-4a84-b13c-adbcf8ca4640
/networking-vpp/nodes/c45-control-2/ports/8bb3104b-f268-402a-800a-348bdf5a0882
/networking-vpp/nodes/c45-control-3
/networking-vpp/nodes/c45-control-3/ports
/networking-vpp/nodes/c45-control-3/ports/084973a8-3990-4a84-b13c-adbcf8ca4640
/networking-vpp/nodes/c45-control-3/ports/d2f40e95-c57d-4122-a74e-0c1c81a0e455
Compared to neutron port-list:
+--------------------------------------+-------------------------------------------------+-------------------+---------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+-------------------------------------------------+-------------------+---------------------------------------------------------------------------------------+
| 084973a8-3990-4a84-b13c-adbcf8ca4640 | | fa:16:3e:a3:dd:d5 | {"subnet_id": "f3a717c1-527c-4e5a-b2b9-41d161b1b0bd", "ip_address": "172.26.233.200"} |
| 6409b006-b256-4353-abd9-bd2582488def | HA port tenant 3184346d0875454fbb83e394de04982f | fa:16:3e:b5:20:c6 | {"subnet_id": "3d2fd088-b8a4-4a4f-82f5-251c58b65c10", "ip_address": "169.254.192.10"} |
| 8bb3104b-f268-402a-800a-348bdf5a0882 | HA port tenant 3184346d0875454fbb83e394de04982f | fa:16:3e:9b:4b:29 | {"subnet_id": "3d2fd088-b8a4-4a4f-82f5-251c58b65c10", "ip_address": "169.254.192.7"} |
| d2f40e95-c57d-4122-a74e-0c1c81a0e455 | HA port tenant 3184346d0875454fbb83e394de04982f | fa:16:3e:dc:7f:0a | {"subnet_id": "3d2fd088-b8a4-4a4f-82f5-251c58b65c10", "ip_address": "169.254.192.5"} |
+--------------------------------------+-------------------------------------------------+-------------------+---------------------------------------------------------------------------------------+

So we will see port 084973a8-3990-4a84-b13c-adbcf8ca4640 is created on all three controller nodes. , but the binding:host_id only shows c45-control-2.
[root@c45-mgmt yicwang]# neutron port-show 084973a8-3990-4a84-b13c-adbcf8ca4640
+-----------------------+---------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | c45-control-2 |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | plugtap |
| binding:vnic_type | normal |
| created_at | 2017-02-28T07:47:02Z |
| description | |
| device_id | e5cb36fc-fc40-40a8-9e8f-af8fe0fbb8b1 |
| device_owner | network:router_gateway |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "f3a717c1-527c-4e5a-b2b9-41d161b1b0bd", "ip_address": "172.26.233.200"} |
| id | 084973a8-3990-4a84-b13c-adbcf8ca4640 |
| mac_address | fa:16:3e:a3:dd:d5 |
| name | |
| network_id | 47730fc8-7a3a-45e4-aece-5adefcb88737 |
| port_security_enabled | False |
| project_id | |
| revision_number | 8 |
| security_groups | |
| status | ACTIVE |
| tenant_id | |
| updated_at | 2017-02-28T07:47:08Z |
+-----------------------+---------------------------------------------------------------------------------------+

2. Clear the gateway from OpenStack API/Horizon, and the corresponding port is only removed from its binding:host_id, but not from all three. From etcd:
/networking-vpp/nodes/c45-control-1
/networking-vpp/nodes/c45-control-1/ports
/networking-vpp/nodes/c45-control-1/ports/084973a8-3990-4a84-b13c-adbcf8ca4640
/networking-vpp/nodes/c45-control-1/ports/6409b006-b256-4353-abd9-bd2582488def
/networking-vpp/nodes/c45-control-2
/networking-vpp/nodes/c45-control-2/ports
/networking-vpp/nodes/c45-control-2/ports/8bb3104b-f268-402a-800a-348bdf5a0882
/networking-vpp/nodes/c45-control-3
/networking-vpp/nodes/c45-control-3/ports
/networking-vpp/nodes/c45-control-3/ports/084973a8-3990-4a84-b13c-adbcf8ca4640
/networking-vpp/nodes/c45-control-3/ports/d2f40e95-c57d-4122-a74e-0c1c81a0e455

So questions:
(1) Are we expected to create the port on all three controllers for external gateway Neutron port?
(2) If yes, the bug is about we should delete all three ports from etcd when port is being deleted. If no, the bug is about we only need to create the port on on one of the controller.

[Ian Wells (ijw-ubuntu)]

This has gone away as a consequence of fixing the unbind bug - it was an unbind/cleanup issue.