[RFE] VXLAN multicast groups distribution in linuxbridge

[jirikotlin]

Enable creation of VXLANs with different multicast addresses allocated by VNI-address mappings.

Dictionary of multicast addresses and corresponding VXLAN VNI IDs should be loaded from settings.

Option vxlan_group supports multiple addrresses via CIDR, but our goal is to have the ability to control vni-multicast address distribution somehow, not randomly, not straightforward mapping.

Usable to not flood whole network when managing routers between more datacenters and can not use L2population because VXLAN points to external device (outside OpenStack).

Real example:

2 data centers connected with VLANs, both with Cisco ASR-9K on L3.

Routers have its own vni-multicast group mappings:

Router(config-if)# member vni 6010-6030 multicast-group 225.1.1.1
Router(config-if)# member vni 8000-9030 multicast-group 225.2.2.1
etc.

So we can create network with vni i.e. 6011 and it will get right multicast address 225.1.1.1 or vni 8000 with vni 225.2.2.1 etc.

We also need to create networks on demand and not configure routers at each change.

[Armando Migliaccio (armando-migliaccio)]

Can you elaborate a bit more on the existing linuxbridge driver limitation and how you intend to leverage config settings to address the issue you're facing?

[jirikotlin (jirka-v)]

Hi,

currently you can set only one VXLAN group via vxlan_group option from settings. Sometimes you have infrastructure in more data centers and the ability to define multiple VXLAN groups can be very useful in practice – for example to manage routers only in concrete data center and not in whole network. We would need this feature in situations when some of VXLANs points to external device and therefor L2population is not usable.

Changes in code should be trivial, multiple approaches are possible and here is one of them:

In settings /etc/neutron/plugins/ml2/ml2_conf.ini are already present vni_ranges and vxlan_group. We should redefine vxlan_group or maybe better register new setting, for example mcast_ranges.

[ml2_type_vxlan]
vni_ranges = 3333:3336, 4444:4999, 5555:6666

vxlan_group = 224.0.0.10

mcast_ranges = 224.0.0.10: 3333:3336, 224.0.0.11: 4444:4999, 224.0.0.12: 5555:6666

[vxlan]
l2_population = False

In neutron/plugins/ml2/drivers/linuxbridge/agent/common/config.py define mcast_ranges variable as dictionary:

cfg.MultiOpt('mcast_ranges',
 item_type=types.Dict(),
 help=_("Comma-separated list of <multicast address>:<vni_min>:<vni_max> triples "
   "describing how to assign address to VXLAN according its VNI ID. ")),

Last thing is to modify function get_vxlan_group in neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py

def get_vxlan_group(self, segmentation_id):
    mcast_ranges = cfg.CONF.ML2_TYPE_VXLAN.mcast_ranges
    try:
        mcast_addr = None
        for ranges in mcast_ranges:
            for addr, range in ranges.iteritems():
                range = range.split(":")
                if int(range[0]) <= segmentation_id <= int(range[len(range) - 1]):
                    mcast_addr = addr
                    break
        net = netaddr.IPNetwork(mcast_addr)

    except (IndexError, TypeError):
        net = netaddr.IPNetwork(cfg.CONF.VXLAN.vxlan_group)

    # Map the segmentation ID to (one of) the group address(es)
    return str(net.network + (int(segmentation_id) & int(net.hostmask)))

Now it tries to map the segmentation_id (vni) to corresponding address from mcast_ranges, if it fails default vxlan_group is used instead.

Later if we create VXLAN with segmentation_id:

neutron net-create test1 --provider:network_type vxlan –provider:segmentation_id=4444

It gets corresponding multicast address (224.0.0.11).

This is just an initial design and one of many possible approaches, for discussion...

If it will be desired, I would like to provide more information about real use case, write the blueprint, work on patch etc.

Thank you very much for your feedback.

[Armando Migliaccio (armando-migliaccio)]

Thanks for providing more details, we should try and get eyes from linuxbridge experts on this proposal. There may be a design rationale why the vxlan group was considered a single attribute

[Sean M. Collins (scollins)]

The code that was pasted in would break everyone who is not using this new mcast_ranges setting. We would need to preserve the behavior of vxlan_group since not everyone is using different multicast groups per VNID, and introduce this new setting for those who wish to utilize.

[Sean M. Collins (scollins)]

Otherwise I think this is a valid RFE and would be good to have in LB

[jirikotlin (jirka-v)]

Hi Sean, no it wont break agent to people who do not have this in settings - if mcast_ranges is not set, it raises exception and runs the original code with vxlan_group

except (IndexError, TypeError):
        net = netaddr.IPNetwork(cfg.CONF.VXLAN.vxlan_group)

I have tested it right now again to be sure.

(+ linuxbridge agent has in common config DEFAULT_VXLAN_GROUP = '224.0.0.1' so vxlan_group is allways present)

thanks for your feedback

[Carl Baldwin (carl-baldwin)]

Discussed briefly in Drivers' meeting: http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-06-02-22.00.log.html#l-114

tl;dr: Please see Armando's comment (#3). A post to the mailing could spark some discussion with linux bridge experts and possibly expose the reason it was originally considered a single attribute.

[jirikotlin (jirka-v)]

Hi Carl,

I wrote to OpenStack-dev mailing list right now:
http://lists.openstack.org/pipermail/openstack-dev/2016-June/096643.html

regards

[jirikotlin (jirka-v)]

Real example:

2 data centers connected with VLANs, both with Cisco ASR-9K on L3.

Routers have its own vni-multicast group mappings.

Router(config-if)# member vni 6010-6030 multicast-group 225.1.1.1

So we can create network with vni i.e. 6011 and it will get right multicast address.

We also need to create networks on demand and not configure routers at each change.

[jirikotlin (jirka-v)]

Hi,

Kevin Benton pointed out in mailing list that linuxbridge agent already supports more vxlan_groups (using CIDR as described in settings help.)

I have updated title of this RFE and also updated description. Our goal is to have the ability to control vni-multicast address distribution somehow, not straightforward mapping.
We need own distribution of addresses according to vni.

Anyway implementing this feature should not be a problem (because linuxbridge agent already supports multiple addresses).

Maybe there will be next hints from other interested people.

If not, what shall I do next?

[Sean M. Collins (scollins)]

I still think the title of this is not accurate.

What about

"RFE - Allow multiple VNIs to share the same multicast group"

[jirikotlin (jirka-v)]

Hi,
"Allow multiple VNIs to share the same multicast group" is not accurate because when you have single vxlan_group in settings, all VNIs got the same address.

Multiple groups is also implemented, only thing we need is a control over address allocation depending on VNI.

[Sean M. Collins (scollins)]

If you provide a prefix for vxlan_group - as Kevin states in http://lists.openstack.org/pipermail/openstack-dev/2016-June/096645.html - you then get a 1:1 mapping between a VNI and a multicast group.

Hence, my suggestion to change the RFE for this bug to be more descriptive. You are looking to have multiple VNIs share the same multicast group. Your own example in https://bugs.launchpad.net/neutron/+bug/1579068/comments/9 cites a mapping of VNIs to a single multicast group.

[jirikotlin (jirka-v)]

Hi,

yes sorry I have updated the description, It was not accurate according to last discussions. Also added example into description.

Problem is that there are more vni-vxlan_group mappings in router so we need control over address allocation. Allow multiple VNIs to share the same multicast group is already implemented.

Is that clear now? Any other suggestions?

thank you very much

[Assaf Muller (amuller)]

http://eavesdrop.openstack.org/irclogs/%23openstack-meeting/%23openstack-meeting.2016-06-16.log.html#t2016-06-16T22:16:49

The summary is that this RFE has been approved to move forward with a config option. If and when multicast optimization becomes real for OVS, we will be able to consider an API then. Until then, it is specific to the LB agent, and a config option that resolves this RFE makes sense.

[jirikotlin (jirka-v)]

Hi,
should I do something more? Can you set the Milestone and assign me to submit this patch?

Thank you.

[Sean M. Collins (scollins)]

Submit a patch, and place the following in your commit message, and this bug will automatically be updated with the information.

Closes-Bug: #1579068

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/333379

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/333379
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/333379
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8a596f35bb70a2c8ab48f68327662310141bb518
Submitter: Jenkins
Branch: master

commit 8a596f35bb70a2c8ab48f68327662310141bb518
Author: Jiri Kotlin <email address hidden>
Date: Thu Jun 23 14:04:07 2016 +0000

    VXLAN multicast groups in linuxbridge

    Enable creation of VXLANs with different multicast addresses allocated
    by VNI-address mappings. Dictionary of multicast addresses and
    corresponding VXLAN VNI IDs should be loaded from settings. Usable to
    not flood whole network when managing routers between more datacenters
    and can not use L2population because VXLAN points to external device.

    Co-Authored-By: Kevin Benton <email address hidden>
    DocImpact: VXLAN addresses used by linux bridge can be specified per VNI
    Closes-Bug: #1579068
    Change-Id: I24f272ccd6d61d9fa7ea3b6f256fabd381f5434a

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b2 development milestone.