[Neutron DVR] No north-south connectivity after ban l3 agent on node with snat

[Georgy Dyuldin]

Detailed bug description:

After ban l3 agent with 'pcs resource ban neutron-l3-agent <node with snat>' there is no ping from instances to 8.8.8.8 (and any other external IP)

Steps to reproduce:

1. Deploy MOS 9.0 with 3 controllers, 2 computes, 1 cinder nodes, Neutron VLAN + DVR
2. Create net1, subnet1
3. Create DVR router router1, set gateway and add interface to net1
4. Boot vm in net1
5. Check that ping 8.8.8.8 available from vm
6. Find node with snat for router1:
   ip net | grep snat-<id_router> on each controller
7. Ban agent on node from previous step:
   pcs resource ban neutron-l3-agent node-x.domain.tld
8. Wait some time while snat is rescheduling
9. Check that snat have moved to another controller
10. Check that ping 8.8.8.8 available from vm

Expected results:

All pings are ok

Actual result:

There is no success ping at step 10

Reproducibility:

Always

Description of the environment:

- Versions of components: MOS 9.0 build #443
- Network model: Neutron VLAN + DVR

Additional information:

One of ports for net1 is DOWN:

root@node-1:~# neutron port-show 1c4a0c8a-a971-4bfd-882d-d602f230875c
+-----------------------+------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | node-4.test.domain.local |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2016-06-07T08:19:23 |
| description | |
| device_id | 0cfad0ef-bdef-4d16-b4af-c7b380f4d3c8 |
| device_owner | network:router_centralized_snat |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "fa3f1a1b-48d5-4ea3-a787-042fa2bf72da", "ip_address": "192.168.1.4"} |
| id | 1c4a0c8a-a971-4bfd-882d-d602f230875c |
| mac_address | fa:16:3e:9a:13:a8 |
| name | |
| network_id | 18b3b2d0-d40f-41c7-953c-b5cd96dd26f9 |
| port_security_enabled | False |
| security_groups | |
| status | DOWN |
| tenant_id | |
| updated_at | 2016-06-07T08:21:18 |
+-----------------------+------------------------------------------------------------------------------------+

[Oleg Bondarev (obondarev)]

Corresponding upstream bug filed: https://bugs.launchpad.net/neutron/+bug/1590041

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0/mitaka
Change author: Oleg Bondarev <email address hidden>
Review: https://review.fuel-infra.org/21781

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/21781
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 40043c8cc3ec6fb3248b93ea4c1edcb012f7c8eb
Author: Oleg Bondarev <email address hidden>
Date: Tue Jun 7 16:56:17 2016

Revert "DVR: Clear SNAT namespace when agent restarts after router move"

This reverts commit b9103329855481b1415a07e6b31206ba35dabc7d.

Closes-Bug: #1590041
Closes-Bug: #1589936
Change-Id: I9c0c2289e09ff2ac62dd6417d2f8fcf323ef1340

[Kristina Berezovskaia (kkuznetsova)]

Verify on
[root@fuel ~]# shotgun2 short-report
cat /etc/fuel_build_id:
 461
cat /etc/fuel_build_number:
 461
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6349.noarch
 fuel-misc-9.0.0-1.mos8451.noarch
 python-packetary-9.0.0-1.mos140.noarch
 fuel-bootstrap-cli-9.0.0-1.mos285.noarch
 fuel-migrate-9.0.0-1.mos8451.noarch
 rubygem-astute-9.0.0-1.mos750.noarch
 fuel-mirror-9.0.0-1.mos140.noarch
 shotgun-9.0.0-1.mos90.noarch
 fuel-openstack-metadata-9.0.0-1.mos8741.noarch
 fuel-notify-9.0.0-1.mos8451.noarch
 nailgun-mcagents-9.0.0-1.mos750.noarch
 python-fuelclient-9.0.0-1.mos325.noarch
 fuel-9.0.0-1.mos6349.noarch
 fuel-utils-9.0.0-1.mos8451.noarch
 fuel-setup-9.0.0-1.mos6349.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8741.noarch
 fuel-library9.0-9.0.0-1.mos8451.noarch
 network-checker-9.0.0-1.mos74.x86_64
 fuel-agent-9.0.0-1.mos285.noarch
 fuel-ui-9.0.0-1.mos2717.noarch
 fuel-ostf-9.0.0-1.mos935.noarch
 fuelmenu-9.0.0-1.mos273.noarch
 fuel-nailgun-9.0.0-1.mos8741.noarch

neutron+vxlan+dvr, 3 controller and 2 compute nodes

Repeat steps several times. All work correctly

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/newton
Change author: Oleg Bondarev <email address hidden>
Review: https://review.fuel-infra.org/33850

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 11.0/ocata
Change author: Oleg Bondarev <email address hidden>
Review: https://review.fuel-infra.org/34198

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: mcp/ocata
Change author: Oleg Bondarev <email address hidden>
Review: https://review.fuel-infra.org/34926

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/34198
Reason: we do not need 11.0/ocata anymore - use mcp/ocata instead

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexander Ignatov <email address hidden> on branch: mcp/newton
Review: https://review.fuel-infra.org/33850
Reason: Not required for Newton as well as Ocata.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexander Ignatov <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34926
Reason: Not required for Newton as well as Ocata.