HA networks are not removed from EOS in certain circumstances

[Benoît Knecht]

This happens on Openstack Kilo, with networking-arista installed from git master (1aa15e8).

Here's the relevant output in /var/log/neutron/neutron-server.log:

2015-09-10 11:13:56.474 19952 INFO networking_arista.ml2.arista_ml2 [req-b90dc709-f285-4df1-81fe-782d2507b033 ] Executing command on Arista EOS: ['enable', 'configure', 'cvx', 'service openstack', 'region regionOne', u'tenant cda858c828c9ed4052add92415682aa1', u'network id 64b3a3a5-9a42-4a83-9436-d371005ae896 name "HA network tenant 3b5acfe5891d53679cfab751c1bde1cb"', 'segment 1 type vlan id 1079', 'no shared', 'exit', 'exit', 'exit', 'exit', 'exit', 'exit']
2015-09-10 11:18:25.234 19952 INFO networking_arista.ml2.arista_ml2 [req-29dbe45d-0349-4ebd-b815-0b0c612f7670 ] Executing command on Arista EOS: ['enable', 'configure', 'cvx', 'service openstack', 'region regionOne', u'tenant 413856f63356cef4663c60adcbde83b9', u'no network id 64b3a3a5-9a42-4a83-9436-d371005ae896', 'exit', 'exit', 'exit', 'exit', 'exit']
2015-09-10 11:18:25.530 19952 ERROR networking_arista.ml2.arista_ml2 [req-29dbe45d-0349-4ebd-b815-0b0c612f7670 ] Error (1000, u"CLI command 7 of 12 'no network id 64b3a3a5-9a42-4a83-9436-d371005ae896' failed: could not run command") while trying to execute commands None on EOS 172.16.0.7

The first line shows the HA network being created when the HA router is created in Neutron by the "admin" user logged in the "admin" tenant (cda858c828c9ed4052add92415682aa1) for the "tenant0" tenant (3b5acfe5891d53679cfab751c1bde1cb):

neutron --os-username admin --os-tenant-name admin router-create --ha True --tenant-id 3b5acfe5891d53679cfab751c1bde1cb

The second and third lines show the ml2 driver trying (and failing) to remove the HA network from EOS, because the request was initiated from another tenant (413856f63356cef4663c60adcbde83b9). Instead of looking for the network in the tenant where it was created (cda858c828c9ed4052add92415682aa1), it looks for it in the tenant that initiated the request (413856f63356cef4663c60adcbde83b9).

This is due (I think) to the following logic in create_network_precommit and delete_network_postcommit in mechanism_arista.py (HA networks have the particularity of having an empty tenant_id, so the condition below will be true):

tenant_id = network['tenant_id']
if not tenant_id:
    tenant_id = context._plugin_context.tenant_id

If the context is different during network creation and deletion, the tenant_id will be wrong in the end.

[Shashank Hegde (hegde-shashank)]

Hi Benoit,

Can you let me know when the network is being deleted? Neutron does not delete the HA network even when the last HA router is deleted (https://bugs.launchpad.net/neutron/+bug/1367157).

[Benoît Knecht (benoit-knecht)]

Hi Shashank,

You're right, it isn't removed automatically by Neutron, even when the last HA router is removed. In the case above, we tried to remove the HA network manually from the Horizon dashboard. I'm guessing Neutron might try and remove the HA network if the tenant is being deleted, but I haven't checked that in any way.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229576
Committed: https://git.openstack.org/cgit/openstack/networking-arista/commit/?id=3ac3b2e354fa2d3d615f7a46b171feaf273c181b
Submitter: Jenkins
Branch: master

commit 3ac3b2e354fa2d3d615f7a46b171feaf273c181b
Author: Shashank Hegde <email address hidden>
Date: Wed Sep 30 11:25:44 2015 -0700

    Using 'INTERNAL-TENANT-ID' as the network owner

    If a network does not belong to any tenant (such as the HA router network),
    instead of relying on the tenant_id in the plugin context, use a special tenant
    id called 'INTERNAL-TENANT-ID' as the tenant_id in the pluin context can be
    different during creation and deletion depending on the tenant performing the
    operation.

    Closes-Bug: 1494302
    Change-Id: I139c8a4754fcf75f9549a0c2b67697ad311fa4c8