Csnat port missing after update router gateway to another network

[Quan Tian]

When updating router gateway to another external network, original csnat ports are deleted in db, but no new ports are created.
Besides, stale devices belong to the original network remains in the snat namespace.

How to reproduce:

- create two external network, one internal network, one distributed router
- add the internal network interface to the router, set the router's gateway to the first external network
- set the router's gateway to another external network, csnat ports will be deleted in db, stale devices "qg-XXX" will be found in snat namespace.

$ neutron router-port-list test -c id -c mac_address -c fixed_ips -c device_owner
+--------------------------------------+-------------------+--------------------------------------------------------------------------------------+--------------------------------------+
| id | mac_address | fixed_ips | device_owner |
+--------------------------------------+-------------------+--------------------------------------------------------------------------------------+--------------------------------------+
| 233f23b2-7cda-415b-80f3-27d1a014518f | fa:16:3e:b3:8d:4b | {"subnet_id": "c1918178-4697-4de7-9c92-ff618ae8a6ee", "ip_address": "172.16.1.252"} | network:router_gateway |
| 91e2e5f2-30d7-4c1d-9fe8-b42bfab2b67e | fa:16:3e:3e:dc:60 | {"subnet_id": "0537a1e9-a30c-433e-b034-023b74916821", "ip_address": "192.168.100.1"} | network:router_interface_distributed |
+--------------------------------------+-------------------+--------------------------------------------------------------------------------------+--------------------------------------+

$ sudo ip netns exec snat-c618e83d-0ab9-48fc-8efa-a25b7b947978 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
42: sg-49fc22d0-cd: <BROADCAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:ad:29:1d brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.9/24 brd 192.168.100.255 scope global sg-49fc22d0-cd
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fead:291d/64 scope link
       valid_lft forever preferred_lft forever
46: qg-d4ce7b19-cb: <BROADCAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:c2:bc:45 brd ff:ff:ff:ff:ff:ff
    inet 192.168.200.8/24 brd 192.168.200.255 scope global qg-d4ce7b19-cb
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fec2:bc45/64 scope link
       valid_lft forever preferred_lft forever
47: qg-233f23b2-7c: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether fa:16:3e:b3:8d:4b brd ff:ff:ff:ff:ff:ff
    inet 172.16.1.252/24 brd 172.16.1.255 scope global qg-233f23b2-7c
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feb3:8d4b/64 scope link
       valid_lft forever preferred_lft forever

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/413721

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/413724

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/413724
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dcd4c8b8de4972b8e5aff55e041df54f66ac81b3
Submitter: Jenkins
Branch: master

commit dcd4c8b8de4972b8e5aff55e041df54f66ac81b3
Author: Quan Tian <email address hidden>
Date: Wed Dec 21 20:40:08 2016 +0800

    DVR: delete stale devices after router update

    After the external network change of router gateway, the l3 agent
    deletes stale external devices in the qrouter namespace only. This
    doesn't apply to distributed router.

    This patch extract a method named _delete_stale_external_devices from
    RouterInfo and override it in DvrEdgeRouter.

    Change-Id: I2602efd5be92ec57410e54cf9c26641330a52ee6
    Partial-Bug: #1651813

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/413721
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=38c070bde41262b428a384ce156b4835e6162c74
Submitter: Jenkins
Branch: master

commit 38c070bde41262b428a384ce156b4835e6162c74
Author: Quan Tian <email address hidden>
Date: Tue Dec 20 23:39:54 2016 +0800

    DVR: fix csnat port missing after router update

    When updating router gateway to another external network, original csnat
    ports are deleted in db, but no new ports are created because the
    _create_snat_interfaces_after_change method did't take the network
    change into account.

    To reduce the complexity of operation taken by l3 agent caused by the
    csnat ports change, we don't need to delete original csnat ports.

    This patch will skip the csnat port deletion if it is a network change.

    Change-Id: Id8e89daa339424bb3c5efdffd0a59cfed005bb5d
    Closes-Bug: #1651813

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/421989

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/421989
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e5fff0b5fb77dff88fab674c04093a45fc86e4eb
Submitter: Jenkins
Branch: stable/newton

commit e5fff0b5fb77dff88fab674c04093a45fc86e4eb
Author: Quan Tian <email address hidden>
Date: Wed Dec 21 20:40:08 2016 +0800

    DVR: delete stale devices after router update

    After the external network change of router gateway, the l3 agent
    deletes stale external devices in the qrouter namespace only. This
    doesn't apply to distributed router.

    This patch extract a method named _delete_stale_external_devices from
    RouterInfo and override it in DvrEdgeRouter.

    Conflicts:
            neutron/agent/l3/dvr_edge_router.py
            neutron/agent/l3/router_info.py
            neutron/tests/unit/agent/l3/test_agent.py

    Change-Id: I2602efd5be92ec57410e54cf9c26641330a52ee6
    Fixes-Bug: #1651813
    (cherry picked from commit dcd4c8b8de4972b8e5aff55e041df54f66ac81b3)