[RFE] Allow DVR for E/W while leaving N/S centralized

[Swaminathan Vasudevan]

Use Case
========
OpenStack is deployed in an L3 fabric so the external network cannot be extended to all compute nodes. Even though this means SNAT and floating IP traffic (North/South) will be run through a network node with external network access, the operator still wants the east/west routing offload offered by DVR.

So even though the topology does not allow for the N/S DVR direct routing, we want to have a way to still take advantage of the E/W direct routing.

Potential Solution
==================

Provide a Configurable option to configure Floatingips for DVR based routers to reside on Compute Node or on Network Node.
Also proactively check the status of the agent on the destination node and if the agent health is down, then configure the Floatingip on the Network Node.

Provide a configuration Option in neutron.conf such as

DVR_FLOATINGIP_CENTRALIZED = 'enforced/circumstantial'

If DVR_FLOATINGIP_CENTRALIZED is configured as 'enforced' all floatingip will be configured on the Network NOde.
If the DVR_FLOATINGIP_CENTRALIZED is configured as 'circumstantial' based on the agent health the floatingip will be configured either in the compute node or on the Network Node.

If this option is not configured, the Floatingip will be distributed for all bound ports and for just the unbound ports the floatingip will be implemented in the Network Node.

[Armando Migliaccio (armando-migliaccio)]

This RFE sounds like one of the proposals touted in bug 1583694. Can you elaborate on the differences?

I personally think that relying on a global option is a bad design choice, and I expressed this time and time again. A more effective solution would be to let this behavior be API-driven.

[Swaminathan Vasudevan (swaminathan-vasudevan)]

The proposal above is with a configuration option where customer if required want to have all their floatingip functionality centralized and only have the east-west routing using DVR then they can go with this global config option. (DVR_FLOATINGIP_CENTRALIZED = 'enforced')

If (DVR_FLOATING_CENTRALIZED = 'circumstantial') then while the scheduler tries to schedule the Floatingip it would check for the health of the compute node agent and if for either reason that the agent is dead or the agent is not capable of handling the floatingip, then it would be started at the centralized node.

The bug 1583694 handles just the allowed_address_pair cases where there is no port-binding for the private port that has the floatingip assigned, this is irrespective of the config option.

[Armando Migliaccio (armando-migliaccio)]

OK, thanks for the explaination. That aside, you are really describing a solution, but what is the problem you are trying to solve? The extra knob/complexity must be justified by a sensible use case.

[Oleg Bondarev (obondarev)]

One of possible use cases is when there is no public connectivity on compute nodes: we still want to benefit from DVR for east-west traffic, while all north-south goes through centralized/network/gateway nodes.

[Armando Migliaccio (armando-migliaccio)]

If there's no public connectivity on compute nodes one *can't* use a key feature for DVR, such as distributed DNAT. So I wonder: why would a deployer have DVR enabled and yet prevent the compute nodes from accessing external access? Makes hardly any sense to me for the complexity involved.

[Swaminathan Vasudevan (swaminathan-vasudevan)]

Yes if there is no local public access in the compute node the distributed DNAT feature can't be used. I think that's the reason having an option to schedule DNAT in the centralized node in such circumstances (where the public network access is not available or the agent is dead on that node) will allow the instances to be reached from external network.

[Oleg Bondarev (obondarev)]

>> why would a deployer have DVR enabled and yet prevent the compute nodes from accessing external access?

In case of L3 fabric underlying network currently there is no simple way to create a single provider/external network and connect all compute nodes to it. Here an option would be to have several dedicated nodes for external connectivity.
In this case having internal routing directly from computes is another key feature of DVR, especially for the cases where not much external traffic is required.

[Armando Migliaccio (armando-migliaccio)]

This was discussed during today's drivers meeting [1]. One proposal that came out of the meeting is to leverage the existing agent_mode (whose current acceptable values are 'dvr', 'dvr_snat', 'legacy') to control scheduling and agent behavior to force centralized DNAT, rather than bring in yet another obscure config knob. Please see meeting logs for more details.

[1] http://eavesdrop.openstack.org/meetings/neutron_drivers/2017/neutron_drivers.2017-07-06-22.00.log.html

[Swaminathan Vasudevan (swaminathan-vasudevan)]

dvr_snat agent mode is currently used in the network_node. I don't think we should be changing that agent_mode to accomodate the centralized floating_ips?

In that case, I am assuming that you are suggesting that we should come up with say for example: 'dvr_no_fip' agent mode, which when configured on compute hosts will not create any fip. And so the scheduler should be intelligent enough to figure out if the port is binded to a host with 'dvr_no_fip' agent, then the floating_ip creation should be notified only to the 'dvr_snat' agent host.

[Armando Migliaccio (armando-migliaccio)]

@Swami: correct, that's the suggestion made during the drivers meeting.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/485333

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/485333
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9515c771e742a5b6d29b17f84f49a0b39706489b
Submitter: Jenkins
Branch: master

commit 9515c771e742a5b6d29b17f84f49a0b39706489b
Author: Swaminathan Vasudevan <email address hidden>
Date: Wed Jul 19 12:13:43 2017 -0700

    DVR: Provide options for DVR North/South routing centralized

    DVR supports both East/West and North/South routing. While the
    SNAT is centralized the DNAT is mostly distributed. There are
    certain circumstances where the DNAT might be centralized when
    the ports are unbound.

    In order to have a well defined behavior and when there are
    no external network connectivity available in the compute host,
    the DNAT functionality is centralized.
    In order to achieve this we are introducing a new agent type
    option 'dvr_no_external' to centralize the DNAT.

    This new L3 agent type ('dvr_no_external') would only allow the East/West
    routing to occur in the compute host and the DNAT or Floating IP will be
    configured in the centralized network node.

    Change-Id: Ia5d7336e478e0fa5ba62b7ae5ed0c56656116d94
    Partial-Bug: #1667877

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/493321

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/493321
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5524bf8434e6e81213859abd6490ab44cbbd21ec
Submitter: Jenkins
Branch: stable/pike

commit 5524bf8434e6e81213859abd6490ab44cbbd21ec
Author: Swaminathan Vasudevan <email address hidden>
Date: Wed Jul 19 12:13:43 2017 -0700

    DVR: Provide options for DVR North/South routing centralized

    DVR supports both East/West and North/South routing. While the
    SNAT is centralized the DNAT is mostly distributed. There are
    certain circumstances where the DNAT might be centralized when
    the ports are unbound.

    In order to have a well defined behavior and when there are
    no external network connectivity available in the compute host,
    the DNAT functionality is centralized.
    In order to achieve this we are introducing a new agent type
    option 'dvr_no_external' to centralize the DNAT.

    This new L3 agent type ('dvr_no_external') would only allow the East/West
    routing to occur in the compute host and the DNAT or Floating IP will be
    configured in the centralized network node.

    Change-Id: Ia5d7336e478e0fa5ba62b7ae5ed0c56656116d94
    Partial-Bug: #1667877
    (cherry picked from commit 9515c771e742a5b6d29b17f84f49a0b39706489b)