backport: ARP entries dropped by DVR routers when the qr device is not ready or present

[Roman Rufanov]

Please backport: ARP entries dropped by DVR routers when the qr device is not ready or present - https://bugs.launchpad.net/neutron/+bug/1501086
Customer found on MOS 7.0

[Roman Podoliaka (rpodolyaka)]

According to https://bugs.launchpad.net/neutron/+bug/1501086 this is fixed in Mitaka - closing as Invalid for 9.0

[Alexey Stupnikov (astupnikov)]

There is a fix merged to upstream stable/kilo branch https://review.openstack.org/237769. I am going to test and cherry-pick it.

[Oleg Bondarev (obondarev)]

Need to backport to 8.0 once https://review.openstack.org/#/c/238989/ is merged

[Alexey Stupnikov (astupnikov)]

I have set bug Status for MOS 7.0 to Incomplete since I couldn't reproduce it on our lab. Roman, could you please provide me steps-to-reproduce for this issue?

[Sergey Belous (sbelous)]

https://review.openstack.org/#/c/238989/ is merged.
Let's wait for next sync with stable/liberty.

[Alexander Ignatov (aignatov)]

Patch is being polished on the upstream, the expectation is that it will be fixed before HCF

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Oleg Bondarev <email address hidden>
Review: https://review.fuel-infra.org/16602

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/16602
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: f82049048bb2eb959000f531ad8bd91dd2c3ed24
Author: Oleg Bondarev <email address hidden>
Date: Tue Feb 2 07:55:16 2016

Merge the tip of origin/stable/liberty into origin/openstack-ci/fuel-8.0/liberty

Note: commit ed7ad25 Revert "Revert "Revert "Remove TEMPEST_CONFIG_DIR in the api tox env"""
sets min tox version to 2.3.1 while we currently use 1.9.2.
This patch sets it back to 1.9.2 in order for tests to pass.
Will be reverted back to 2.3.1 once https://bugs.launchpad.net/fuel/+bug/1540516
is fixed.

8476f6f Add relationship between port and floating ip
0bd401c DVR: optimize check_ports_exist_on_l3_agent()
9246cff Change check_ports_exist_on_l3agent to pass the subnet_ids
a133de3 Keep reading stdout/stderr until after kill
ed7ad25 Revert "Revert "Revert "Remove TEMPEST_CONFIG_DIR in the api tox env"""
05f8099 Ensure that tunnels are fully reset on ovs restart
b908c55 Update HA router state if agent is not active
aebd27f Resync L3, DHCP and OVS/LB agents upon revival
8e685c8 Fix floatingip status for an HA router
80c9e84 DVR:Fix _notify_l3_agent_new_port for proper arp update
c12bf81 Fix L3 HA with IPv6
2298566 Make object creation methods in l3_hamode_db atomic
0cc889f Cache the ARP entries in L3 Agent for DVR
8bde9c4 Cleanup veth-pairs in default netns for functional tests
2b96f42 Do not prohibit VXLAN over IPv6
1ab1e58 Fix get_subnet_for_dvr() to return correct gateway mac
3b42dee Imported Translations from Zanata
ca193d0 Revert "Change function call order in ovs_neutron_agent."
96d4ab3 Remove check on dhcp enabled subnets while scheduling dvr
f5299d3 Check gateway ip when update subnet
0d5d7c7 Add tests that constrain db query count
8fb3f9d Don't call add_ha_port inside a transaction
a370fa3 Log INFO message when setting admin state up flag to False for OVS port
bf92dbb DVR: notify specific agent when deleting floating ip
99d1c0d Call _allocate_vr_id outside of transaction
2468b3d Move notifications before DB retry decorator
1b609d2 DVR: handle dvr serviceable port's host change
ad75ccc Imported Translations from Zanata
2e6e135 Run functional gate jobs in a constrained environment
6902c87 DVR: notify specific agent when creating floating ip
00b800d Tox: Remove fullstack env, keep only dsvm-fullstack
d11e9cb Force L3 agent to resync router it could not configure
42f4332 Support migrating of legacy routers to HA and back
4d85fa1 Updated from global requirements
f175cd7 ML2: Add tests to validate quota usage tracking
79d4a08 test_migrations: Avoid returning a filter object for python3
745b546 Do not autoreschedule routers if l3 agent is back online
430892a Avoid full_sync in l3_agent for router updates
fa9fba2 In port_dead, handle case when port already deleted
1d8aff3 Add compatibility with iproute2 >= 4.0

Conflicts:
 neutron/db/l3_dvr_db.py
 neutron/db/l3_hamode_db.py
 neutron/tests/functional/services/l3_router/test_l3_dvr_router_plugin.py
 neutron/tests/unit/agent/l3/test_agent.py

Closes-Bug: #1496341
Closes-Bug: #1531244
Closes-Bug: #1527581
Closes-Bug: #1528201
Closes-Bug: #1528207
Closes...

[Roman Podoliaka (rpodolyaka)]

Fix merged ^

[Kristina Berezovskaia (kkuznetsova)]

Verify on:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "529"
  build_id: "529"
  fuel-nailgun_sha: "baec8643ca624e52b37873f2dbd511c135d236d9"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "e2d79330d5d708796330fac67722c21f85569b87"
  fuel-ostf_sha: "3bc76a63a9e7d195ff34eadc29552f4235fa6c52"
  fuel-mirror_sha: "fb45b80d7bee5899d931f926e5c9512e2b442749"
  fuelmenu_sha: "e071216cb214e34b4d861478033425ee6a54a3be"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "a365f05b903368225da3fea9aa42afc1d50dc9b4"
(neutron+vxlan)

Reproduced on #478 iso.

Steps:
1) Create heat stack with yaml-file (https://paste.mirantis.net/show/1833/) and parameters (https://paste.mirantis.net/show/1834/): heat stack-create TESTA -f test.yml -e par.yml
2) Check l3-agent logs with message: Cannot find device "qr-
There are no such messages on iso #529

[Roman Podoliaka (rpodolyaka)]

Returning back to Confirmed, otherwise, we should have closed it as Invalid after a month without feedback...

Roman R., please work with the Maintenance team to close this properly.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Swaminathan Vasudevan <email address hidden>
Review: https://review.fuel-infra.org/22596

[Alexey Stupnikov (astupnikov)]

7.0 patch is ready to be merged. This patch was included in upstream stable/kilo and agreed to merge to MOS7.0 by core reviewer Oleg Bondarev.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/22596
Submitter: Denis V. Meltsaykin <email address hidden>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: f0630177b1780af97a422c21a82b1feca052859e
Author: Swaminathan Vasudevan <email address hidden>
Date: Mon Jun 27 08:01:30 2016

Cache the ARP entries in L3 Agent for DVR

There seems to be a timing issue between the
ARP entries that arrive from the server to
the agent and the internal qr-device getting
created by the agent.
So those unsuccessful arp entries are dropped.

This patch makes sure that the early ARP entries
are cached in the agent and then utilized when
the internal device is up.

Closes-Bug: #1531244
Change-Id: I9ec5412f14808de73e8dd86e3d51593946d312a0
(cherry picked from commit d9fb3a66b4aead65fac2df08a5f34538e7af4d7b)

[Ekaterina Shutova (eshutova)]

Verified on MOS 7.0 + mu5 updates.

Fix is uploaded to the nodes.
Checked as in comment #10. No such messages related to qr-device in l3-agent.log.