[RFE]Support l4 udp protocol loadbalance

[zhaobo]

Currently, neutron-lbaas or octavia implemented the haproxy just support TCP,HTTP,HTTPS,TERMINATED_HTTPS.
We need support udp if we want the data flow lb towards this kind protocol or requirements.

Usecases
===========
Syslog which is a logging standard that is used by many applications, also users can send the log info to the central log server based on UDP 514. For deploy env on OpenStack, users may want configure the customer’s vms to send the log the log server, for a big scale, the loadbalancer for UDP is very important for a stable udp traffic and low bandwidth use. The central log server could get the all log information and analysis in next step.

IoT(Internet of Things), most protocols running on IoT network are based on UDP protocol, such as CoAP, DDS. UDP is also emerging as one of the protocols of choice for new IoT applications due to its low bandwidth requirements. Applications such as push info in publish-subsribe model , High-speed traffic for real-time applications. Currently, the IoT udp loadbalancer act on the physical devices, but with the development of Cloud,more and more customers wish the exist traditional devices can migrate/move to cloud, this is very huge requirements for make the virtual function like devices if possible.

Currently, Octavia have a strong binding with haproxy which have a good support on L7 traffic loadbalance, and the backend amp with the reference haproxy driver just support TCP protocol, this will limit the steps to other areas. For most vendors who maintain some high concurrency website, such as Real-time video chats, Video on-line, also include IoT. All of them request the udp loadbalance. (This is very high level sight).

For the general simple request, we need the udp traffic can forward to the specific UDP server, just like the tcp loadbalance now,(VIP:PORT to NODEX:PORTX).
Point a: This is the most simple for this, just like I said above, SEE [1]

Point b：For the video stream on-line services, we can run RTP/RTCP/RTSP protocol. The main intent is like the figure below[2]. It’s like the Real-Time meeting services.
The left side are the meeting initiators in the internet, they want to some internal people to take part in the meetings. So they need to send the video traffic to the specific public IP which locate on the loadbalancer.
Then loadbalancer forward this traffic to the specific dispatcher service. Dispatcher service also mutil-copy/forward the traffic to the network its connected. In this network, all the machines which run a meeting application will receive the udp traffic. SEE [2]

[1] http://paste.openstack.org/show/624458/
[2] http://paste.openstack.org/show/624459/

[Michael Johnson (johnsom)]

Can you provide some use cases as to why you would like UDP and what UDP protocols you would use this for?

[zhaobo (zhaobo6)]

@Michael Johnson
Hi， michael. Thanks for comments here.

1.As for IOT，the most part of the protocols used by services/app is based on udp. Such as CoAP （Constrained Application Protocol), DDS（Data Distribution Service for Real-Time Systems.
IOT need the capbility of real-time, large scaled data prosession...But it doesn't care about the reliable transmission.
So the udp in L4 is the right one.
CoAP protocol works for wire-less IOT cases.
DDS is based on the publish-subscribe model, and it is good to apply to the IOT view which need the requirements about High-speed source data access in application layer, interests subcribtion and etc.

2.DNS protocol

3.video chatting: RUDP（Reliable User Datagram Protocol）, RTP(RealTime Protocol), UDT(UDP-based Data Transfer Protocol).

So our servers behind vip need the udp support for this view. Currently, LB API doesn't support the udp value into the protocol parameters. I think it must be the haproxy not support udp loadbalance.

Also, the real openstack product may support mutil-backend drivers, such as LVS , nginx and haproxy. So I think we could add the udp support API, then if the backend driver support udp, we can make some desciptions.

[Miguel Lavalle (minsel)]

Today I had an IRC conversation with johnsonm about this RFE:

14:57:36 mlavalle | johnsom: ping
14:57:44 johnsom | o/
14:58:13 mlavalle | hey, any chance this could be discussed next week:
                                 | https://bugs.launchpad.net/octavia/+bug/1657091
14:58:14 johnsom | mlavalle What is up?
14:58:15 openstack | Launchpad bug 1657091 in octavia "[RFE]Support l4 udp protocol loadbalance"
                                   | [Wishlist,Incomplete] - Assigned to zhaobo (zhaobo6)
14:58:45 mlavalle | johnsom: some of my co-workers in China are very interested in this
14:59:37 johnsom | Sure. We are really looking for some solid use cases to bump the priority of
                                   | this up. DNS is not a valid UDP load balancer usecase IMHO....
15:00:32 mlavalle | johnsom: ok, I'll put a note to this effect in the bug and let them come up with
                                 | a better use case
15:00:52 johnsom | mlavalle There are a couple of options of how to add this, would be interested to
                                     | know if there are preferences or protocol inspection requirements
15:01:21 xgerman_ | mostly we will a non-haproxy lb need
15:01:26 johnsom | mlavalle Is there resource available to develop if we come up with a plan?
15:01:57 mlavalle | johnsom: I cannot answer that right now, but I will also make a note about it in
                                 | the bug
15:02:06 johnsom | Yeah, there are options to add it to our existing amp plus options to come up
                                   | with an alternate amp all together
15:02:23 johnsom | Ok
15:02:47 mlavalle | johnsom: do you want us to bring this up in a specific session next week?
15:04:19 johnsom | mlavalle Yeah, let's add it as a topic here:
                                   | https://etherpad.openstack.org/p/octavia-ptg-queens (no one had brought it up or
                                   | added it yet). I will find a slot on the agenda
15:05:08 mlavalle | johnsom, xgerman_ Thanks guys. See you next week :-)

I also added this topic to the Octavia Denver PTG etherpad

[Michael Johnson (johnsom)]

FYI, this bug has been migrated to Storyboard here: https://storyboard.openstack.org/#!/story/1657091

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/503606

[zhaobo (zhaobo6)]

As Micheal's kind suggestion during PTG, then I test LVS simple driver with amp in my env, it works for the some simple udp loadbalance function in the currently deployment type. But there is a problem now, LVS driver can not meet the most part of Octavia data models. So I think need to extend the amp agent RESTAPI and inject some LVS CLI code to agent(will find some other python library which support LVS operation). Then find a good way to how to advance the LVS part. Also I still need to collect the real requirements from users..

[zhaobo (zhaobo6)]

@ALL, please check the use case again. Thank you.

[zhaobo (zhaobo6)]

I update the RFE decription.

[Michael Johnson (johnsom)]

zhaobo Please use the StoryBoard bug: https://storyboard.openstack.org/#!/story/1657091
This bug is no longer used

[Gregory Thiemonge (gthiemonge)]

Abandoned after re-enabling the Octavia launchpad.