I have three controller nodes and the Neutron servers on these controllers are set behind Pacemaker and HAProxy to realize active/active HA using DevStack. MariaDB Galera cluster is used as my database backend.I am using the latest codes.
I have one external network and one router. If I run $neutron router-gateway-set at three controllers at the same time, I will end up with three ports created on the external network. Although, only the latest created port would be updated as the router's default gateway IP, the former two ports would remain in both routerports and ports database.
Besides, the former two ports could not be deleted from $ neutron router-gateway-clear command. They can only be deleted from database by hand.
How to reproduce:
Step 1: Create an external network
$ neutron net-create ext-net --router:external True
Step 2: Create a subnet on the external network
$ neutron subnet-create --name ext-subnet ext-net 192.168.122.0/24
Step 3: Create a router
$ neutron router-create router-gateway-test
Step 4: Set gateway to the router on three controllers at the same time
On controller1:
$ neutron router-gateway-set router-gateway-test ext-net
On controller2:
$ neutron router-gateway-set router-gateway-test ext-net
On controller3:
$ neutron router-gateway-set router-gateway-test ext-net
The port list on the router after the above commands could be seen here http://paste.openstack.org/show/484091/
Step 5: Clear the gateway on the router
$ neutron router-gateway-clear router-gateway-test ext-net
The port list on the router after Step 5 could be seen here http://paste.openstack.org/show/484092/
As we can see, the two ports created earlier were not able to be cleared through CLI.
Step 6: Try to deletes the rest two router ports
$ neutron port-delete 3095887a-cb2d-46eb-bf56-be6305596868
$ neutron port-delete 76a42eda-33ce-4048-882a-6f8cb4d7137c
where '3095887a-cb2d-46eb-bf56-be6305596868' and '76a42eda-33ce-4048-882a-6f8cb4d7137c' are the two remaining gateway ports on the router
The result is shown here http://paste.openstack.org/show/484094/
The routerports and ports database info could be seen here http://paste.openstack.org/show/484095/ , where '9a261e95-1f3b-4c8f-91a6-098b9fab7c25' is the id of the router we created in Step 3.
=========================================================================
Update on 1/27
I have tested this bug again on all-in-one DevStack setup with one neutron-server and api_workers > 0, which is the default DevStack setup.
Replace Step 4 as
$ neutron router-gateway-set router-gateway-test ext-net & neutron router-gateway-set router-gateway-test ext-net
and you will receive the same result as what I received in multi-controller setup.
Marked as low because I just don't see this being much of a a problem in practice. I can't think of why 3 different people/processes will be interested in updating a router's gateway all at the same time.