[RFE] Allow operator control of "on-link" routes for subnets in the same Neutron network

Bug #1668145 reported by Harald Jensås
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Wishlist
Kevin Benton

Bug Description

When adding multiple subnets on a single network the dhcp agent will set dhcp-options to advertise "on-link"/"same-segment" classless static routes to all the subnets.

A couple of use cases where these on-link/link-local routes are undesirable:

a) When using Ironic for baremetal provisioning the baremetal nodes might be on a different L2 broadcast segment with a DHCP-relay.

b) In a Spine-Leaf deployed Openstack, a pattern is to use identical VLAN id's for provider networks in each leaf. Same VLAN id, but different L2 domain and different IP subnets.

IMO, creating these routes are a bit opinionated. If we don't create them by default, the operator/end-user is fully able to create them if they are desired on a per-subnet basis. But since we decided these "should" be there, the operator loose the ability to control this.

Changed in neutron:
assignee: nobody → Harald Jensås (harald-jensas)
status: New → In Progress
Revision history for this message
Harald Jensås (harald-jensas) wrote :

Fix proposed in review:

  https://review.openstack.org/#/c/438171/

Changed in neutron:
status: In Progress → New
Changed in neutron:
assignee: Harald Jensås (harald-jensas) → nobody
summary: - Allow the end user control of "on-link" routes for subnets in the same
+ Allow operator control of "on-link" routes for subnets in the same
Neutron network
summary: - Allow operator control of "on-link" routes for subnets in the same
+ [RFE] Allow operator control of "on-link" routes for subnets in the same
Neutron network
tags: added: l3-ipam-dhcp rfe
Revision history for this message
Kevin Benton (kevinbenton) wrote :

If you have different subnets on the same network in different broadcast domains, that should be represented in the data model via subnets bound to different segments. Then we can automatically distinguish on the DHCP agents which routes should be on-link routes.

Revision history for this message
Harald Jensås (harald-jensas) wrote :

I am aware of the new segments model, even tried to play with it but it is scarcely documented as of yet. It is possible to model this using the segments, however:

a) Segments cannot be safely backported. This could?
b) Segments do not currently work with DHCP-relay. This approach would.

Revision history for this message
Harald Jensås (harald-jensas) wrote :

Hi, I would like to follow up.

Source: https://tools.ietf.org/html/rfc3442

  Local Subnet Routes

     In some cases more than one IP subnet may be configured on a link.
     In such cases, a host whose IP address is in one IP subnet in the
     link could communicate directly with a host whose IP address is in a
     different IP subnet on the same link. In cases where a client is
     being assigned an IP address on an IP subnet on such a link, for each
     IP subnet in the link other than the IP subnet on which the client
     has been assigned the DHCP server MAY be configured to specify a
     router IP address of 0.0.0.0.

The rfc says, quote:
  "the DHCP server MAY be configured to specify a router IP address of 0.0.0.0."

I would like to see that _MAY be_ available when I use the Linux DHCP agent.

Changed in neutron:
assignee: nobody → Harald Jensås (harald-jensas)
status: New → In Progress
Revision history for this message
Akihiro Motoki (amotoki) wrote :

If I understand correctly, what you need is a way to disable advertisement of local static routes from DHCP agent, right?
I haven't understood how neutron "segments" feature does not work for you.

> a) Segments cannot be safely backported. This could?

In General, any new feature cannot be backported according to the OpenStack backport policy.

Do you mean you backport something by yourself?

> b) Segments do not currently work with DHCP-relay. This approach would.

I think this depends on what DHCP implementation is provided for a subnet.
DHCP relay just forwards DHCP requests with unicast communication.
What is a problem if local subnet routes is advertised? Are you thinking your own DHCP server?
I might be missing something.

> Source: https://tools.ietf.org/html/rfc3442
...
> The rfc says, quote:
> "the DHCP server MAY be configured to specify a router IP address of 0.0.0.0."

"0.0.0.0" in this case just means a corresponding subnet is reachable directly.
Do you mean this can be disabled perhaps?

Revision history for this message
Akihiro Motoki (amotoki) wrote :

The status of RFE bug under discussion needs to be Confirmed or Triaged.

Changed in neutron:
status: In Progress → Confirmed
importance: Undecided → Wishlist
Revision history for this message
Harald Jensås (harald-jensas) wrote :

Correct, I would like the advertisements of local static routes from DHCP to be disabled.

I don't see how Neutron Segments would solve that as there is no code in DHCP agent that takes the segments into consideration.
 Ref: https://bugs.launchpad.net/neutron/+bug/1669594

My usecase is the following:
- Tripleo deployment on routed datacenter networks.
- Ironic BMaaS on routed datacenter networks.

For this to work, I need a DHCP server that can server clients reaching the DHCP server via DHCP-relay configure on ToR switches. To enable this, I belive two changes is required:
 1. The static routes must be optional.
 2. Need a way to tell DHCP agent to only create a single port, on one subnet.

Using the Routed Provider Networks, and Neutron Segments will require that I place a DHCP agent in each L2 Segment. It is obviously not practical to deploy openstack services in each and every rack of a datacenter just to be able to deploy openstack, it is also not very cost effective to place a system running a DHCP agent for every 20-30 baremetal systems.

In the TripleO usecase there is also the issue that there is no OS::Neutron::Segment Heat resource yet.

Don't get me wrong regarding Neutron Segments and the routed provider networks support, this is a great feature but for my usecase the advanced scheduling etc is not necessary.

Let me share my opinion specifically on the local static routes:
 These static routes have _no_ place in the DHCP agent implementation.
 Neutron should not automatically create them at all.
 The end user can add "host-routes" on a Neutron subnets if they want local static routes.

 IF Neutron wants to enforce these, why implement them in the DHCP agent?
  They should rather be added as host-routes on the neutron subnet object instead.
  And IMO the end user should be allowed to decide...

  As it is implemented now, it is DHCP implementation specific which means the
  end user is in for a surprise if they change the DHCP implementation.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/459492

Changed in neutron:
assignee: Harald Jensås (harald-jensas) → Brian Haley (brian-haley)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/459861

Changed in neutron:
assignee: Brian Haley (brian-haley) → Kevin Benton (kevinbenton)
Revision history for this message
Brian Haley (brian-haley) wrote :
Download full text (4.9 KiB)

Just some notes from a discussion on #openstack-neutron

<hjensas> kevinbenton: There is indeed multiple physnets. But, the single node running neutron services is not connected to more than 1 physnet. The nodes in the other physnets are baremetal machines, under Openstack Ironic.
<kevinbenton> hjensas: that's fine, but when the ports are created for those ironic nodes, they should get associated with subnets on different segments
<hjensas> kevinbenton: We need the single machine only connected to one physnet to provide DHCP for Ironic nodes on multiple physnets. The datacenter have the routing infrastructure, and dhcp-relay agents on the routers/ToR switches.
<hjensas> kevinbenton: but I cannot create multiple segments without multiple physical-networks on the neutron node.
<kevinbenton> hjensas: yes you can, just specify different physnets
<kevinbenton> hjensas: that doesn't mean those physnets are on the neutron node
<hjensas> kevinbenton: ah, so I don't need a bridge_mapping with the physnet?
<hjensas> kevinbenton: I can create --physical-network foo ...
<kevinbenton> hjensas: the bridge_mapping is just to tell which physnets a given node can access
<kevinbenton> hjensas: in the neutron server config for [ml2_type_flat], you may need to add physnets there if it's not set to '*'
<hjensas> kevinbenton: yes, I see. I can create multiple segments.
<kevinbenton> hjensas: now the problem you may run into is getting the port actually bound to the correct segment
<kevinbenton> hjensas: IIRC ironic ports are actually just left UNBOUND, right?
<kevinbenton> hjensas: or do they have a mechanism driver loaded?
<hjensas> kevinbenton: well, the next problem is that I get 1 DHCP namespace, with dhcp opts for 1 subnet.
<kevinbenton> hjensas: well how many subnets do you have on the network?
<hjensas> kevinbenton: 3
<kevinbenton> hjensas: and creating segments eliminated the other two?
<kevinbenton> (in DHCP namespace)
<hjensas> kevinbenton: I have 1 network, 3 segments with 1 subnet in each segment. - I get 1 DHCP namespace with 1 port in segment1 - subnet0, and opts for subnet0 only.
<kevinbenton> hjensas: and dhcp is enabled on other 2, right?
<kevinbenton> hjensas: oh, i see the issue
<kevinbenton> hjensas: it looks like we have the assumption built into dhcp scheduler that it won't be able to service remote subnets
<kevinbenton> hjensas: i don't think the dhcp relay use case was considered
<hjensas> kevinbenton: http://paste.openstack.org/show/607919/
<hjensas> kevinbenton: but, I do have bridge_mappings for 2 segments. So we should see two namespaces? Or atleast opts for two subnets?
<kevinbenton> hjensas: we should just see opts inside the one namespace
<kevinbenton> hjensas: but that's being explicitly prevented right now
<kevinbenton> hjensas: because of some logic that thinks there is no point in the dhcp agent trying to serve a network it's not attached to
<kevinbenton> hjensas: https://github.com/openstack/neutron/blob/master/neutron/api/rpc/handlers/dhcp_rpc.py#L152-L155
<kevinbenton> hjensas: let me think about this for a bit, we may just want to remove that logic completely to consider dhcp relay
<hjensas> kevinbenton: I created this earlie...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Brian Haley (<email address hidden>) on branch: master
Review: https://review.openstack.org/459492
Reason: Unnecessary after discussion

Changed in neutron:
assignee: Kevin Benton (kevinbenton) → Brian Haley (brian-haley)
Changed in neutron:
assignee: Brian Haley (brian-haley) → Kevin Benton (kevinbenton)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/438171
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=723169245978db29fdd42af18d5958e76c23a0fe
Submitter: Jenkins
Branch: master

commit 723169245978db29fdd42af18d5958e76c23a0fe
Author: Harald Jensas <email address hidden>
Date: Sat Feb 25 11:05:22 2017 +0100

    Only add "on-link" routes for L2 adjacent subnets

    When multiple subnets exist on a single network, the
    DHCP agent adds on-link routes for all of them since
    they are in the same L2 network. If either subnet
    has a segment_id it can only be considered as on-link
    if they match, else we should not include a subnet
    route.

    These extra routes are optional anyways according to
    RFC 3442, but were added for the use case when all of
    the subnets are considered adjacent, which allows
    instances to bypass the router and communicate directly.

    Closes-Bug: #1668145
    Change-Id: Iae889e9226a61059cd4f3d37fbe48d013b7a3482
    Implements: blueprint tripleo-routed-networks-deployment

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 11.0.0.0b2

This issue was fixed in the openstack/neutron 11.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.