Once a Gateway is set L-3 agent attempts to update external Gateway on every router update

[Rajeev Grover]

once an external gateway is set, due to a logic error, for every subsequent router update the L-3 agent incorrectly concludes there is a change in external gateway. This causes the codepath to set external gateway getting invoked un-necessarily.

In process_router(...)

  ....
        ex_gw_port = self._get_ex_gw_port(ri) returns ri.router.get('gw_port')
  ...
          if ex_gw_port and ex_gw_port != ri.ex_gw_port:
            self._set_subnet_info(ex_gw_port) <--------
 ...
         ri.ex_gw_port = ex_gw_port

The _set_subnet_info adds an element to the ex_gw_port thus making it different from the gw_port obtained out of router dict. Any subsequent ex_gw_port != ri.ex_gw_port would result True, incorrectly.

One way to fix it would be to change

From:

         if ex_gw_port and ex_gw_port != ri.ex_gw_port:
To:

        if (ex_gw_port and (not ri.ex_gw_port
                            or ex_gw_port['id'] != ri.ex_gw_port['id'])):

To reproduce:
a) Create a router
b) Add external gateway to it (router-gateway-set <router_name> <external_nw>)
c) Do something that would cause the Plugin to send down an update to the router. This can be an action un-related to the external gateway e.g. add an interface to the router ( router-interface-add <router_name> <subnet>)
d) Besides adding the interface, the agent code would treat this as if router gateway has been added or changed. It re-attempts to plug the router-gateway port and activities associated with it. This can be observed by reading the code and I believe observing the logs. I think you will see gratuitous arps getting sent out. I instrumented the code for my debugging so don’t remember if you will see clear logs without instrumentation or not.
    This definitely contributes to degradation of service due to slowing down things but don’t know yet if there will be a communication interruption or not. It does affect DVR too because DVR has additional actions such as setting up SNAT redirection as part of setting up the external gateway and can cause duplicate entries.

[Kevin Benton (kevinbenton)]

Hi Rajeev,

Are you going to submit a patch for this? If so, please assign the bug to yourself so someone else doesn't start working on it.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/109776

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/109776
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=596908f8a8aecff06b079147b957c7da3dc7a51b
Submitter: Jenkins
Branch: master

commit 596908f8a8aecff06b079147b957c7da3dc7a51b
Author: rajeev <email address hidden>
Date: Fri Jul 25 18:50:34 2014 -0400

    Improve external gateway update handling

    Once gateway is set, external_gateway_added() was getting called every time
     a router update was received. The check for change in external
    gateway compared previously cached copy of gateway port (ri.ex_gw_port) with
     the one passed in through update router (ri.router['gw_port']).
    The cached copy was already being modified by code so the two values would
    always appear to be different.
    Making the change to compare correctly and remove actions not required
    for gateway update.

    Change-Id: I1a703b327e6c569dfaa8263a222e4bc797e5dbfd
    Closes-Bug: 1348737