Race between deleting last HA router and a new HA router API call

[LIU Yulong]

During the delete of tenant last HA router, neutron will also delete the HA network which can be racy if a new HA router API call is coming concurrently.

Some known exceptions:
1. NetworkNotFound: (HA network not found when create HA router HA port)

2. IpAddressGenerationFailure: (HA port created failed due to the
   concurrently HA subnet deletion)

3. DBReferenceError(IntegrityError): (HA network was deleted by
   concurrently operation, e.g. deleting the last HA router)

4. HA Network Attribute Error
   http://paste.openstack.org/show/490140/

Consider using the Rally to do the following steps to reproduce the race exceptions:
1. Create 200+ tenant, each one has 2 or more user
2. Create ONLY 1 router for each tenant
3. Concurently do the following:
  (1) one user try to delete the LAST HA router
  (2) other user try to create some HA router

[Miguel Angel Ajo (mangelajo)]

https://review.openstack.org/#/c/265682/

[LIU Yulong (dragon889)]

Patch:
https://review.openstack.org/#/c/265682/

[Ann Taraday (akamyshnikova)]

@LIU Yulong (dragon889)
I filed a separate bug about IpAddressGenerationFailure as I didn't find one https://bugs.launchpad.net/neutron/+bug/1562887

[OpenStack Infra (hudson-openstack)]

Change abandoned by LIU Yulong (<email address hidden>) on branch: master
Review: https://review.openstack.org/265682
Reason: See the alternate patch:
https://review.openstack.org/#/c/282876/

[John Schwarz (jschwarz)]

3 of the 4 original issues in the first post are now fixed, and the one that isn't is addressed by a separate bug report:

1. NetworkNotFound: fixed by the introduction of _create_ha_interfaces_and_ensure_network

2. IpAddressGenerationFailure: https://bugs.launchpad.net/neutron/+bug/1562887

3. DBReferenceError: Opened a separate bug, https://bugs.launchpad.net/neutron/+bug/1533460, and fixed by https://review.openstack.org/#/c/260303/

4. HA Network Attribute Error: fixed by the introduction of _create_ha_interfaces_and_ensure_network

I think this bug can be closed.

[Jakub Libosvar (libosvar)]

As per comment #5 - this bug was split into smaller pieces addressed by different bug numbers. Closing as Invalid per John's request.

[John Schwarz (jschwarz)]

The "_create_ha_interfaces_and_ensure_network" has been introduced in https://review.openstack.org/#/c/282876/ (forgot to link the original patch, sorry :))