Support L3-only networking

[Nell Jerram]

This RFE bug describes and proposes a type of Neutron network in
which connectivity between the VMs attached to that network is
provided by L3 routing. This type of network provides full (subject
to security policy) IP connectivity between VMs in that and other
routed networks: v4 and v6, unicast and multicast; but it provides no
L2 capability, except as required for this IP connectivity, plus
correct operation of the ICMP, ARP and NDP protocols that exist to
support IP. Therefore, this kind of network is suitable for VMs that
only communicate over IP.

Why would anyone want that? Compared to the other kinds of networks
that provide connectivity at L2, its arguable benefits are that:

- it is conceptually simpler, in that VM data is transported in a
  uniform way between a VM and its compute host, between compute
  hosts, and between the data center network and the outside world,
  without any encapsulation changes anywhere

- as a practical consequence, it is easier to debug, using standard
  tools such as ping, traceroute, wireshark and tcpdump

- its scale is not limited in the way that VLAN-based and VXLAN-based
  networks are, by the practical diameter of the physical underlying
  L2 network.

FYI I started proposing/discussing this as a devref at https://review.openstack.org/#/c/198439/, and lots more detail can be found there about how I think this could work. However, I understand that that is not the correct process, hence in principle starting again here as an RFE bug.

NOTE: The above mentioned devref has been abandoned in favor of a spec here: https://review.openstack.org/#/c/238895/

[Paul Carver (pcarver)]

Our use case isn't identical, but we (AT&T) also see a need for routed networks because there are many NFV/VNF types where L3 interconnects are more appropriate than L2 interconnects.

[Carl Baldwin (carl-baldwin)]

neiljerram, kevinbenton, and I discussed [1] how this relates to [2]. The way I see it, this rfe proposes L3 to the port. L2 is confined to the port and used only to bootstrap the VM on to the L3 network by providing responses to DHCP, arp, the like.

The tap interface is not bridged to a bridge. Therefore, dnsmasq must be configured to listen to tap interfaces directly.

Having read the devref [3], I think this proposal is somewhat consistent with [4] in that they will use provider types to mark a network.

[1] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2015-07-09.log.html#t2015-07-09T16:28:42
[2] https://bugs.launchpad.net/neutron/+bug/1458890
[3] https://review.openstack.org/#/c/198439/
[4] https://review.openstack.org/#/c/196812/

[Kyle Mestery (mestery)]

Marking confirmed since Carl and I have seen it. It's not approved yet, I suggest we focus on this at next week's drivers meeting.

[Nell Jerram (neil-jerram)]

Thanks, Kyle. I will be available for the drivers meeting, and appreciate your suggestion to look at this during that meeting. FYI I also just emailed the list with a complete latest status on this work: http://lists.openstack.org/pipermail/openstack-dev/2015-July/069937.html.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Neil Jerram (<email address hidden>) on branch: master
Review: https://review.openstack.org/197578
Reason: This interface driver is now planned to be in networking-calico, rather than in the core Neutron tree, and hence this patch submission is abandoned.

[Armando Migliaccio (armando-migliaccio)]

I am unclear on the current state of this request, can you refresh our memory?

[Paul Carver (pcarver)]

There is actually some discussion going on between a number of parties on whether it makes sense to extend Neutron to support routed networks or if a new project should be started such that Nova can choose between multiple port providers. Personally I haven't finalized my opinion on the topic, but there is a case being made that Neutron's data model is so tightly tied to layer 2 bridging that it doesn't make sense to try to bend it to support layer 3.

I've put in a vBrown Bag session on the topic http://sched.co/4Rmx at the summit and we're working on scheduling some time at the summit, but separate from the official summit schedule, for additional discussion.

If the sentiment runs towards not supporting L3 networks in Neutron then this RFE bug would be closed. However, I think it's too soon to say. There is also be a case to be made that supporting L3 networks in Neutron would be better than having Neutron only provide the L2 networks.

[Nell Jerram (neil-jerram)]

Although I'm intrigued by 'Gluon' - and will plan to come to your vBrownBag, Paul - I'd prefer to continue exploring this within Neutron. I think it will be better if we can devise a single integrated API (i.e. Neutron) that can describe both existing and routed networking semantics, and I don't believe that any of the core people currently discussing are opposed to this. We are just working through possible models and details to find the clearest model.

[Nell Jerram (neil-jerram)]

Armando, in answer to your query: I think it makes sense for this bug to remain open as a current RFE. Certainly it is still something that I (any my team) would like Neutron to support.

The current focus of work in this area is https://review.openstack.org/#/c/225384/7, and I am actively contributing to that. However, your query has prompted me to realize that it may not be clear or explicit enough whether this RFE is one of the use cases that that spec is addressing. I will check this with Carl.

One more clarification that might not be obvious to you: the remaining work here is all about having semantics correctly described on the Neutron API. You may also have seen my recent networking-calico announcement, saying that one can now use networking-calico with vanilla Liberty to set up a cluster with routed (in the Calico sense) networking. That is true in a practical sense, but the resulting semantics do not match the semantics that someone might expect for the sequence of Neutron API requests that were made to set it up. So that is what we'd now like to address.

[Paul Carver (pcarver)]

Neil, I'm not convince I know the right answer. Think of the vBrownBag as a fact finding session. I wanted to make sure there was something on the schedule because I was worried that the limited number of Neutron design sessions would prevent this topic from getting an official timeslot to discuss. I did add related topics to the design session Etherpad, but put in the vBrownBag proposal as a fallback. It looks like the Thursday 14:40 design session may be able to touch on this topic, but probably will need to cover a lot of other territory as well.

[Kyle Mestery (mestery)]

I think it's pretty clear people want L3 networks in Neutron, so lets solve it there. I'm not sure it makes sense to add an entirely new project to solve that single use case. Updating the model to support this makes the most sense, and I'd like to see Carl's work here land which would enable it.

[Armando Migliaccio (armando-migliaccio)]

I believe that Carl's work differs from this one. The choice of words is poor in the sense that both talk about routing and it may lead to misunderstanding easily.

We will have summit time about this topics, let's make sure we can have a productive one so that we nail this once and for all, whilst preserving backward compatibility.

Neil: if in the meantime you can provide an update on that devref https://review.openstack.org/#/c/198439/ or respin in light of the Liberty enhancement, that would be great.

[Paul Carver (pcarver)]

I also believe that Carl's work differs from this. Also Calico isn't quite the same thing either. There are certainly concepts in common and I think it's useful to discuss, but there's also potential for confusion because of similar (or identical) terminology being interpreted differently based on context of datacenter vs telco/WAN/VNF, and depending on context of overlay vs underlay.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/238895

[OpenStack Infra (hudson-openstack)]

Change abandoned by Neil Jerram (<email address hidden>) on branch: master
Review: https://review.openstack.org/198439
Reason: This is now superseded by https://review.openstack.org/#/c/238895/, and I would appreciate if interested reviewers would review and comment on that new version.

https://review.openstack.org/#/c/238895/ continues the conversation as a Neutron spec - which is appropriate as there are interesting spec-level details to debate.

Thanks - Neil

[Nell Jerram (neil-jerram)]

Armando - following your comment at https://bugs.launchpad.net/neutron/+bug/1472704/comments/12, and Kyle's querying whether the use cases of this bug and https://bugs.launchpad.net/neutron/+bug/1458890 are the same or different, I've respun and updated https://review.openstack.org/198439 as a Neutron spec - https://review.openstack.org/#/c/238895/ - and tried to clearly answer all of those questions. (Both in my own head, and for everyone else!)

Would be very happy to discuss this with you and any other interested parties in Tokyo!

[Kyle Mestery (mestery)]

Thanks Neil, this is great! Looking forward to spending some time with you on this in Tokyo.

[Armando Migliaccio (armando-migliaccio)]

Based on the latest comments on [1], is it fair to say that we can track this initiative in the context of [2,3]? To this aim, can I mark this duplicate of [2]?

[1] https://review.openstack.org/#/c/238895/
[2] https://bugs.launchpad.net/neutron/+bug/1458890
[3] https://blueprints.launchpad.net/neutron/+spec/routed-networks

[OpenStack Infra (hudson-openstack)]

Change abandoned by Neil Jerram (<email address hidden>) on branch: master
Review: https://review.openstack.org/238895
Reason: It looks very likely now that the semantic needs of the networking-calico backend - i.e. for a way of describing a network that only guarantees L3 connectivity - will be provided by Carl's spec at https://review.openstack.org/#/c/225384/. Hence abandoning this spec, to remove any impression that it might be competing with Carl's work.