routed provider networks limit to one host
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Fix Released
|
Wishlist
|
Unassigned |
Bug Description
There seems to be limitation for a compute node to only have interface on one segment in a multisegment network. This feels wrong and limits the compute resources since they can only be part of one segment.
The purpose of multi segment networks is to group multiple segments under one network name. i.e. operators should be able to expand the IP pool without having to create multiple network for it like internet1, internet2, etc.
The way it should work is that a compute node can belong to one or more segments. It should be up to the operator to decide how they want to segment the compute resources or not. It should not be enforced by the simple need to add IP ranges to a network.
way to reproduce.
1. configure compute nodes to have bridges configured on 2 segments
2. create a network with 2 segments.
3. create the segments
2018-04-17 15:17:59.545 25 ERROR oslo_messaging.
2018-04-17 15:18:18.836 25 ERROR oslo_messaging.
description: | updated |
tags: | added: rfe |
tags: |
added: rfe-confirmed removed: rfe |
Changed in neutron: | |
importance: | Undecided → Wishlist |
Miguel Lavalle (minsel) wrote : | #1 |
Miguel Lavalle (minsel) wrote : | #2 |
Let's wait a little longer to see if we get a response from the submitter
bjolo (bjorn-lofdahl) wrote : | #3 |
hi all,
Sorry for the delay, busy as ...
Adding IP ranges does not scale. Openstack mentality and design has to be "think big". For a single segment one can have perhaps 2000 IPs, but after that you will probably start seeing broadcast issues, etc. What if an operator needs 10.000 IPs on the same network? Also, not all switch hardware supports to have more than one GW IP per segment(vlan)
The logical structure from an end user perspective should be that there is only one network named "internet". "internet" is made up of one or more segments, and each segment has one or more IP ranges. One can also imagine that each segment is of a different type to provide different types of ports. i.e. sr-iov (vlan), vxlan, etc.
b
bjolo (bjorn-lofdahl) wrote : | #4 |
.... however, this should not enforce the operator to segment the compute resources. It should be independent of that.
In our setup for example. We run a corporate cloud and we have several external networks that are tied into different security zones in our network for various resources. Some networks are router external only, others are provider only, and some are both. Some have very small IP range, while others require very large and growing IP ranges. Access to these networks are handled with RBAC of course.
On the compute side, we have many different flavors of compute like large mem, gpu, AVX512 instruction set, dedicated cores, etc, etc.
With the current limitation with segments, any compute node can only belong to one segment per network which forces us to segment the compute resources as well. This is far from desired.
b
Harald Jensås (harald-jensas) wrote : | #5 |
There is actually a comment in source code that we may want to consider relaxing this in the future to allow multiple segments on one host. (https:/
I don't see why we cannot have one host connect to multiple segments via different bridges. One connection over Ethernet, another over Infiniband or just a different ethernet nic or sr-iov as Bjørn (or Björn?) is describing.
I think if this was allowed we could also find it useful to deploy network nodes with connections to multiple segments and then have segmented compute resources. The one network node could provide services such as DHCP on all the segments. Instead of deploying the network services on some compute nodes or dedicated nodes as we need to do today.
+1 to explore this.
Miguel Lavalle (minsel) wrote : | #6 |
@Bjorn,
What you describe in #3 is exactly the rationale for routed networks: create big networks by isolating the broadcast domains in segments and then stitching those segments together with routers to form the big network. This is reflected in this presentation from the Barcelona Summit: https:/
I find intriguing, though, your comments in #4. Can you elaborate on the limitations imposed by the network segments in the way you deploy your compute hosts? An example of how the network segmentation limits your flexibility in deploying compute hosts would be helpful (to me at least). As Harald points out, the limit of one segment per compute was contingent on better understanding routed network use cases
Miguel Lavalle (minsel) wrote : | #7 |
Hi Bjorn,
Any additional insights you can give us on your use case?
Kris Lindgren (klindgren) wrote : | #8 |
We have a very similar use case.
In the future, I can see the need for wanting to have a compute node that can belong to 2 different routed networks.
However, Our current need/use case is that we want to have a compute node that is connected to multiple segments on the *same* routed network. Our network is an L3 spine+leaf design where the L2 is terminated at the ToR. The need for multiple segments to the same host is driven by our networking teams desire to keep the broadcast domains for the L2 small. Currently, each segment (vlan) has a /22 worth of ip address space, and a segment covers all servers attached to that ToR. The desire is to add additional vlans and thus subnets to the ToR and thus all servers under that ToR, to allow more VM's to be booted on the existing HV's. We need to have anywhere from 3-8 /22's of capacity depending on the number of hosts and the size of the HV's in the racks. You can imagine having ~8000 ip's in the same L2 vlan will start having broadcast domain issues. Hence the desire to break that up into smaller domains. In our clouds that are not yet running routed-networks, we do this without issue just by mapping in additional networks to the compute nodes.
Kris Lindgren (klindgren) wrote : | #9 |
We actually have some code started to attempt to do the above.
We ran into 3 problems.
1.) The artificial limitation that was posted above.
2.) If you are trying to use the same physcal_network (say the switch name) for those segments which are contained on that switch (each segment being configured as type vlan with a unique segmentation_id). The idea is that you want to have both segments use the same interface external interface (eg: br0) and just create the appropriate vlan tag off of that when a port is bound, via the vlan driver. (eg. br.401 and br.402)
The issue that we ran into is the for loop inside: https:/
3.) Once I worked around that, I ran into the issue that the bridge name needs to be based upon the segment_id and not the network_id. Once you have both ports successfully binding using their correct segments, without this change both the tap device and the br0.401 and br0.402 devices are added to the same bridge. Which obviously won't work, and depending on spanning tree config could loop the network or cause other general badness. Additionally, for Operators who are moving to routed networks, if we use the segment_id, we could (in the database) force the segment_id to match the original network_id that we pulled into the routed network. Which means we wouldn't have the issue where all of the existing vm's domain.xml information is wrong for the bridge that the tap devices should be plugged into.
Miguel Lavalle (minsel) wrote : | #10 |
Based on the feedback above, we have several operators / deployers interested in this RFE. Let's bring it up to the attention of the drivers team.
tags: |
added: rfe-triaged removed: rfe-confirmed |
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master) | #11 |
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
assignee: | nobody → David Bingham (wwriverrat) |
status: | New → In Progress |
Miguel Lavalle (minsel) wrote : | #12 |
This RFE was approved to go ahead with a PoC. The drivers team understands that when the functionality is merged in Neutron, the code developers will include devref documentation
tags: |
added: rfe-approved removed: rfe-triaged |
David Bingham (wwriverrat) wrote : | #13 |
Have created a migration script here: http://
Migration would go something like this:
1.) Download: http://
2.) Deploy migration bridges:
Run on all compute nodes: routed_
3.) Deploy code out to computes && HV's
4.) Ping some vm's on one HV
5.) Migrate on that HV:
6.) Verify vm's maintain connectivity
7.) Start pinging a couple different vm's throughout the env
8.) Run bridge migration on remaining HV's:
routed_
9.) Verify vm's maintain connectivity
What the migration script does (high level)
1.) Create a bridge name based off the segment_id that you pass it (openstack network segment list)
2.) Add a veth pair that links the new bridge and the old bridge together. This will provide network connectivity between vm's/and the outbound interface as things are moved around
3.) Move each vm from the network_uuid based bridge to the segment_uuid bridge
4.) Update each vm's domain.xml file in both the /run and normal startup locations
5.) restart libvirt so it loads updated domain.xml files
Nova over the period of a few minutes to hours (depending on the size of the cloud) will update the network_info_cache and store the update bridge_id into its cache.
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-specs (master) | #14 |
Related fix proposed to branch: master
Review: https:/
Slawek Kaplonski (slaweq) wrote : auto-abandon-script | #15 |
This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.
Changed in neutron: | |
assignee: | David Bingham (wwriverrat) → nobody |
status: | In Progress → New |
tags: | added: timeout-abandon |
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master) | #16 |
Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https:/
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.
Slawek Kaplonski (slaweq) wrote : auto-abandon-script | #17 |
This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master) | #18 |
Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https:/
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.
Changed in neutron: | |
status: | New → In Progress |
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master) | #19 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #20 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #21 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #22 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #23 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #24 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #25 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #26 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #27 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #28 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #29 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : | #30 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master) | #31 |
Change abandoned by "Sahid Orentino Ferdjaoui <email address hidden>" on branch: master
Review: https:/
Reason: dup, i removed the change id and create new one by mistake
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #32 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 1bfbc33ce0390ee
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Mon Apr 25 11:31:03 2022 +0200
ovs: handle segmentation ids per network ports
This is changing the datastructure that maintains the relationship
between ports and networks to also handle the segmenation ids related.
This will be necessary in future to support multiple segments per
networks on a same physical provider network.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: Iaf40ddc20692a3
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-specs (master) | #33 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit e562468c6257b5a
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Thu Mar 17 07:55:16 2022 +0100
OVS: multiple routed provider segments per host
The proposed spec is to extend the current feature routed provider
networks to allow provisioning more than one segment per physical
network.
The support is for OVS only.
Related-Bug: #1764738
Related-Bug: #1956435
Change-Id: I00e32b5b4fc6e4
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #34 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 6ec0bc70a7907ab
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Apr 27 09:03:55 2022 +0200
ovs: make vlanmanager to handle more vlan mapping per network
This change is updating the vlanmanager data structure to handle for a
given network more than one vlan mapping. This is a prerequisite work
needed to progress on accepting several segments per network per
host.
The work done here is trying to avoid changing logic in the
current implementation. Unit test should not have value updated,
but probably signatures changed.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: Ic3c147136549b1
OpenStack Infra (hudson-openstack) wrote : | #35 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 7a1e253851dfb9b
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Apr 27 14:20:54 2022 +0200
ovs: use a local vlan per network/
This is using changes introduced before to support for a network more
than one vlan.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: Ifd61e379c3cef3
OpenStack Infra (hudson-openstack) wrote : | #36 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 7ceb935da82ee5d
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Wed Apr 27 18:02:33 2022 +0200
dhcp/rpc: retrieve network details with segments
When segment plugin is enabled, we should return segments details as
they are part of network.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I1dab155bc812f8
OpenStack Infra (hudson-openstack) wrote : | #37 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 81853192480c4d0
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Thu Apr 28 11:30:55 2022 +0200
utils: add option to generate dhcp device id per segmentation
This will be used in future when dhcp will handle different
segmentation ids.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: Ie005285ed66704
OpenStack Infra (hudson-openstack) wrote : | #38 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 6db791ac0fde866
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Thu Apr 28 16:12:24 2022 +0200
ml2/mech: bind port on segment that subnet belongs
If subnet is attached to a segment we want to endure that the bind
will happen on it.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I56b22820d29b2d
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron-fwaas (master) | #39 |
Related fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-fwaas (master) | #40 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 553e6b64110d573
Author: elajkat <email address hidden>
Date: Mon Sep 5 15:40:53 2022 +0200
Adopt to latest VlanManager and oslo.db changes
The work for making routed networks work with multiple segments per
host, introduced new signature for VlanManager.get, requesting
segmentatio
With oslo.db 12.1.0 some unit tests started to fail, with using the
CONTEXT_R/W session we can fix it.
Adopt dsvm-functional target name as [0] changed in Neutron, so the new
name of the target is dsvm-functional
[0]: https:/
Change-Id: Ie7459974f6f235
Related-Bug: #1956435
Related-Bug: #1764738
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #41 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 1d8e3b79dbe9dac
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Fri Apr 29 12:02:45 2022 +0200
dhcp: make device manager to clean only not used devices
This change makes device manager action to clean device a bit smarter
by comparing port registered for a given network with device in live.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I0b6f9b59b94cf0
OpenStack Infra (hudson-openstack) wrote : | #42 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit cb332acb29ec0bc
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Fri Apr 29 17:28:03 2022 +0200
dhcp: add/use cleanup stale devices API
This is adding new API for the dhcp driver to clean stale
devices. Previously it was not necessary since a dhcp port was related
to a nemaspace and when the namespace got deleted, the device was also
removed.
Now with multisegments we can have more than one dhcp port per
namespace based on segmenation id so we should ensure to remove the
stale device.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I4a46b034a5feab
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master) | #43 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #44 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 7c1a894ce5c27a4
Author: elajkat <email address hidden>
Date: Thu Oct 13 14:30:17 2022 +0200
Nit: network_update in ovs_neutron_agent has a bad LOG
The log entry had %(tag)s but the dict has 'segmentation_id' as key,
so let's change tag to segmentation_id.
Change-Id: Ic6e82a31efe779
Partial-Bug: #1956435
Partial-Bug: #1764738
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron-specs (master) | #45 |
Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https:/
Reason: Patch abandoned due to the lack of activity. Please feel free to restore if needed.
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #46 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit d1c2d2c4fee0844
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Mon May 2 16:33:47 2022 +0200
dhcp: support multiple segmentations per network
This change makes the DHPC agent to handle multiple segmentation per
network.
For each segmentation a DHCP Process will be started, this has the
benefit to keep the current logic of building a DHCP service per
network domain.
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I88264ce2303cba
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master) | #47 |
Fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master) | #48 |
Change abandoned by "Sahid Orentino Ferdjaoui <email address hidden>" on branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master) | #49 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit eeff5b3c81ae784
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Fri Aug 5 08:56:27 2022 +0200
db: add segment_index to the unique constraint
For multi segments support we have update the unique contraint so
`segment_index` will be part of it.
Related-Bug: #1791233
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: Ic564131dcd7525
OpenStack Infra (hudson-openstack) wrote : | #50 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit be0996c30850a87
Author: Sahid Orentino Ferdjaoui <email address hidden>
Date: Mon Nov 14 21:58:13 2022 +0100
segment: enable multisegments support for host
This updates the exception with a log message informing that
multi-segments is supported by OVS only at that point.
This also add fullstack tests that validates multisegs deployment on a
physnet.
Closes-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <email address hidden>
Change-Id: I3811a4ca28906d
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master) | #51 |
Related fix proposed to branch: master
Review: https:/
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master) | #52 |
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit e9ce025a69dd00d
Author: yatinkarel <email address hidden>
Date: Thu Feb 16 12:54:22 2023 +0530
[dhcp agent] Fix clean_devices
Exception[1] is seen in periodic_resync of
dhcp, this patch fixes it.
[1] TypeError: DhcpLocalProces
required positional argument: 'network'
Related-Bug: #1764738
Related-Bug: #1956435
Change-Id: Ie265b864718d9e
Brian Haley (brian-haley) wrote : | #53 |
From all the changes that have merged this seems to be complete, will close.
Changed in neutron: | |
status: | In Progress → Fix Released |
@Bjolo,
I am not sure I understand your request. Is your concern the ability to associate more IP pools to an existing network? Why not add more subnets? Why add segments?