L3 agent should support provider external networks

[Robert Kukura]

The l3-agent requires an "external" quantum network, and creates ports on that network, but does not actually use the external quantum network for network connectivity. Instead, it requires manual configuration of an external_network_bridge for connectivity.

Now that the provider network extension allows creation of quantum networks that do provide external connectivity, the l3-agent should support external connectivity through such networks. This can be done in a backward-compatible way by interpreting an empty external_network_bridge configuration variable as indicating that the external network should actually be used for connectivity.

[Salvatore Orlando (salvatore-orlando)]

I agree the external network might benefit from the provider networks extension.

It seems the intention here is to map the external gateway to a physical network if external_network_bridge = ""
So the external gateway, for instance, could be eth0.
Floating IPs and the gateway itself are currently configured as quantum ports on the external network bridge; in this case we'll need to find alternative; configuring multiple IPs on the physical interface is a possibility which is however different from what the agent currently does.

I am thinking whether we can keep the external_network_ bridge and just use the provider network extension to map it to a physical network, or not use it as you propose but instead attach floating IPs to br-int.

[Salvatore Orlando (salvatore-orlando)]

Btw, in my opinion it is too late now for Folsom inclusion - but I'm not the one making the call!

[dan wendlandt (danwent)]

note: we will definitely have to evaluate the scope of this change before deciding whether it can be back-ported to Folsom.

[dan wendlandt (danwent)]

Bob, do you want to target a particular milestone for this?

[dan wendlandt (danwent)]

non-trivial change, unlikely to make g-2

[dan wendlandt (danwent)]

this is really a mini-feature masquerading as a bug, so we need to get this wrapped up ASAP. I think salvatore is planning on making the changes for the NVP plugin's L3 implementation, so this bug should just track the changes for plugins using l3-agents.

[dan wendlandt (danwent)]

Let's talk about this at the meeting today. I think its getting to be too late in the game to make any "mini-feature" changes like this, especially to such a critical code path.

[dan wendlandt (danwent)]

Bob has been doing testing, and it seems like apparently this already works for the linux bridge plugin.

I'll move this to havana in case code changes are needed for any other plugins, otherwise, we should close this bug as invalid.

[Édouard Thuleau (ethuleau)]

Me, I tested that with OVS plugin of the Quantum Folsom release and it works nicely.

I set a physical network and I created a provider network with it (on a specific VLAN) with flags "external" and "shared".
I can use this network to plug directly a VM (flat public network) and I can use it as an external network for routers.

[Robert Kukura (rkukura)]

Lets keep this bug open for now to cover adding devstack support for provider external networks, and also file a corresponding documentation bug. Can devstack updates still get in for grizzly?

[Robert Kukura (rkukura)]

Also, should we change the default value of external_network_bridge to be empty in havana, so that provider networks are used by default, and maybe phase out the external bridge support at some point?

[dan wendlandt (danwent)]

Devstack is not tied to any release, and changes can merge whenever.

The problem with changing the default is that I suspect people with existing deployments that rely on the default value of br-ex would be broken when they upgrade from folsom to grizzly. We can however change the documentation to steer people with new deployments toward using the provider networks model for this.

[Robert Kukura (rkukura)]

I've now verified the l3_agent works fine using a real quantum external network (external_network_bridge set to the empty string in config) with both openvswitch and linuxbridge using devstack. I'm working on a patch for devstack that will serve as the resolution for this bug.

My current plan is to configure this in devstack using the following localrc settings:

EXT_NETWORK_TYPE: external network's type (flat, vlan, local, ...)
EXT_PHYSICAL_NETWORK: external network's physical network name
EXT_SEGMENTATION_ID: external network's segmentation id (VLAN tag, tunnel ID, ...)

If EXT_NETWORK_TYPE is set, then PUBLIC_BRIDGE must not also be set, br-ex will not be configured, and the external network will be configured as a provider network with the specified attributes.

Comments?

Is it worth having devstack optionally setup the host as the external network's gatway?

[Mark McClain (markmcclain)]

Has the devstack landed for this bug?

[Robert Kukura (rkukura)]

On 09/30/2013 11:12 AM, Mark McClain wrote:
> Has the devstack landed for this bug?
>
> ** Tags removed: folsom-backport-potential
> ** Tags added: havana-rc-potential
>
> ** Changed in: neutron
> Milestone: havana-rc1 => None
>

I keep forgetting all about this, and have not added devstack code to
support it. I think its a bit lower priority than the ml2-related
updates and documentation, but I'll try to get to it in the next week or so.

-Bob

[Salvatore Orlando (salvatore-orlando)]

Decreasing priority to medium and setting status to "confirmed" reflecting latest comments on the bug.

[Prakash Kashyap (p2kash)]

I would like to take a shot at completing this mini feature.
If I understand this right, this is a way for l3agent to pass off L3NAT & routing to external network to an external router/network created by the provider. Is this understanding correct?

If my understanding is right, I would like to take a look at what needs to be done. I understand that this has to be done maintaining backward compatibility.

[Carl Baldwin (carl-baldwin)]

Am I correct in saying that the Neutron code now supports using a provider networks for external gateway but that devstack still needs to be updated?

[Robert Kukura (rkukura)]

This bug does not involve using an external router in place of neutron-l3-agent for the NAT and routing. It's just an alternative way for the neutron-l3-agent to connect to external networks. Instead of specifying an external bridge, the normal L2 mechanisms are used to connect to the external network(s).

Neutron has supported provider external networks for a long time. What's new in icehouse is support for multiple external networks, in which case provider external networks must be used. What's still missing is devstack support for using provider external networks.

[Prakash Kashyap (p2kash)]

From what I could gather looking at devstack/lib/neutron, if the following code lines in function “create_neutron_initial_netowrk"(shown below) are bypassed when $PUBLIC_BRIDGE is not set, it won’t bring up the bridge “br-ex” in the host running q-l3. It seems all the required steps needed to associate the l3-agent with user defined “external” network are there in devstack/lib/neutron.

Bypassing these two steps is all that is probably needed in devstack to take care of this bug? Is my assessment correct?

I am assuming users who want to use “external net” for connectivity to external world, would not set PUBLIC_BRIDGE variable in “local.conf”.

Lines of code:

function create_neutron_initial_network {
                 :
                sudo ip addr add $EXT_GW_IP/$CIDR_LEN dev $PUBLIC_BRIDGE
                sudo ip link set $PUBLIC_BRIDGE up
                :

[Cedric Brandily (cbrandily)]

This bug has been solved in trunk[1] (icehouse) and backported to stable/havana[2]: When external_network_bridge and gateway_external_network_id are empty, l3-agent uses provider attributes to deploy the external network

[1] https://review.openstack.org/59359
[2] https://review.openstack.org/68601