Direct support for Octavia LBaaS API

[Volodymyr Litovka]

Dear colleagues,

there are some fundamental issues in neutron-lbass architecture (e.g. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120145.html), which prevents deploying LBaaS infrastructure in an automated fashion (e.g. using Heat), and, while Heat support only neutron-lbaas API, it makes it difficult to use both for maintaining projects with LBaaS functions.

On the other hand, Octavia is already an official LBaaS solution for Openstack (https://governance.openstack.org/tc/reference/projects/octavia.html) and will deprecate the current neutron-lbaas extension starting Queens release (February 2018). As for the clients, neutron-lbaas requires the "neutron" client. The "openstack" client only supports Octavia native API. Neutron client has already been marked deprecated.

So, direct Octavia support in Heat is required in order to support LBaaS functionality starting Queens.

Octavia is a standalone API endpoint that is a super-set of the LBaaS v2 API, meaning it is compatible with the same API that neutron-lbaas implemented. It is advertised in keystone as different endpoint:

+-----------+-----------+--------------+----------------+-----------+-------------------------+
| ID | Region | Service Name | Service Type | Interface | URL |
+-----------+-----------+--------------+----------------+-----------+-------------------------+
| 18862b... | RegionOne | octavia | load-balancer | internal | http://controller:9876/ |
| 8cb62a... | RegionOne | octavia | load-balancer | public | http://controller:9876/ |
| 909e9f... | RegionOne | octavia | load-balancer | admin | http://controller:9876/ |
+-----------+-----------+--------------+----------------+-----------+-------------------------+

The API is documented here: https://developer.openstack.org/api-ref/load-balancer/v2/index.html

Since Octavia API is fully backward compatible with neutron-lbaas, from Heat point of view it is required to use Octavia endpoint (adding "/v2.0" prefix to URL, i.e. "GET /v2.0/lbaas/loadbalancers") instead of Neutron endpoint for LBaaS-related calls, when configured in this way.

Thank you.

[Rabi Mishra (rabi)]

I had looked at it (and did some work) before, but the key issue I didn't find is any answer to is how to migrate the existing heat LBaaSv2 resource users (those who don't use octavia backend ex. use haproxy plugin driver).

AFAIK, when using octavia API you're forced to use Octavia backend (amphora vms that requires at least 1GB of RAM to run effectively). Though I've not checked recently and probably missing something, I don't see any clear migration plan yet.

May be we'll have a separate set of resources for Octavia and deprecate the current ones like we did for lbaasv1->lbaasv2?

The other thing to keep in mind is, we don't use octavia backend with the existing lbaas resources at the gate as a amphora VM per loadbalancer deployed is kind of heavy and difficult to debug.

[Volodymyr Litovka (doka.ua)]

Hi Rabi,

it seems somebody (may be you) already found a perfect solution to the migration issue by introducing parameter 'provider' to OS::Neutron::LoadBalancer object (found in latest documentation and twice in Heat's sources). By default, it points to classic neutron-lbaas extension, if used - to another LBaaS provider. If "provider: octavia", Heat will use Octavia's endpoint for creating this object and derivatives.

In this case, migration issue isn't your business - customers will migrate their LBaaS infrastructure step by step manually, introducing new LBaaS provider for new projects and replacing it in existing projects (so, changing the value will cause object replacement).

And this eliminates need to have two different set of resources, since API is fully compatible except URL (prefix "/v2.0").

Thank you.

[Volodymyr Litovka (doka.ua)]

Hi again,

one more comment: if Heat will support both providers (neutron-lbaas and Octavia) at the same time using parameter "provider", it will open way for seamless migration for projects with existing LBaaS infrastructure:

- create new LoadBalancer(Octavia) set, using same existing backend servers (wich are part of existing pool)
- gradually migrate to new VIP or just migrate an external floating IP to new VIP
- delete old LoadBalancer(Neutron) set

So, customer will be able to switch from old to new implementation when it will be convenient.

Thanks.

[Michael Johnson (johnsom)]

FYI, the /v2.0 exists in neutron-lbaas the same as it does with the Octavia endpoint. This did not change between the two. The only change required between the two is changing the endpoint from neutron to octavia.

Also, on migration, if the load balancer was using the Octavia driver under neutron-lbaas, your resources are already present in the Octavia API. It is only alternate drivers that do not (yet) have a migration path as the octavia driver specification is in it's final reviews. However, since this is not a big change like v1 was, we expect migration tools will be fairly straight forward as drivers move over.

[Rabi Mishra (rabi)]

> It is only alternate drivers that do not (yet) have a migration path as the octavia driver specification is in it's final reviews.

That's exactly the concern I've. Without the similar drivers[1] existing like neutron-lbaas (though I've no clue how many of them are being used and work) being available, how do we expect heat lbaas users migrate? I know it's not an issue for users using the default octavia backend/driver with neutron-lbaas. Are we expecting users using things like haproxy namespace driver move to use the default implementation with octavia?

[1] https://github.com/openstack/neutron-lbaas/tree/master/neutron_lbaas/drivers

[Volodymyr Litovka (doka.ua)]

Hi Rabi,

correct me if I'm wrong. As far as I understand, current implementation
of neutron-lbaas provides (a) an API and (b) a way to use different
drivers  as plugins, which implement LBaaS functionality. For external
systems like Heat these "plugins" are hidden, Heat just calls
neutron-lbaas API, knowing nothing about specifics behind the API.

If this is true, then I guess it's not Heat's or whoever else's business
to know about internals. Octavia team claims to be a complete
replacement to current implementation (including plugin architecture for
3rd party drivers) and sooner or later other vendors will align their
drivers with new architecture.

What makes it easy to migrate is that Octavia is completely separate
standalone controller, which work without any changes to existing
Neutron/LBaaS configuration. Thus, if Heat will provide a way to choose
provider (neutron-lbaas or octavia), then customers will continue to use
neutron-lbaas as long as it will be required, with their specific
drivers (haproxy, F5, A10, etc), gradually migrating to Octavia when
time will come.

In terms of HOT, it can be done in the following way (in the same project) :

# this balancer will be created and maintained using neutron-lbaas
implementation,
# calling Neutron API, which configured to use e.g. F5 driver

   balancer:
     type: OS::Neutron::LBaaS::LoadBalancer
     properties:
       name: balancer
       vip_subnet: { get_resource: lan-subnet }

# this balancer will be created and maintained using Octavia implementation,
# calling Octavia API, which configured to use something else

   balancer_new:
     type: OS::Neutron::LBaaS::LoadBalancer
     properties:
       name: n-balancer
*provider: octavia*
       vip_subnet: { get_resource: lan-subnet }

Thank you.

On 12/12/17 6:56 PM, Rabi Mishra wrote:
>> It is only alternate drivers that do not (yet) have a migration path
> as the octavia driver specification is in it's final reviews.
>
> That's exactly the concern I've. Without the similar drivers[1] existing
> like neutron-lbaas (though I've no clue how many of them are being used
> and work) being available, how do we expect heat lbaas users migrate? I
> know it's not an issue for users using the default octavia
> backend/driver with neutron-lbaas. Are we expecting users using things
> like haproxy namespace driver move to use the default implementation
> with octavia?
>
> [1]https://github.com/openstack/neutron-
> lbaas/tree/master/neutron_lbaas/drivers
>

--
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison

[Rabi Mishra (rabi)]

I would suggest we move this to a ML discussion for everyone to participate.

> Octavia team claims to be a complete replacement to current implementation (including plugin architecture for 3rd party drivers) and sooner or later other vendors will align their drivers with new architecture.

As I mentioned earlier that claim is probably not accurate, when it's about supporting other drivers. IMO, having the api compatibility is not enough. When we're replacing existing heat resources to use Octavia(API) directly we probably have to think about heat lbaas users that don't use octavia. If we provide a new set of resources, then it would probably be ok.

>if Heat will provide a way to choose provider (neutron-lbaas or octavia), then customers will continue to use neutron-lbaas as long as it will be required, with their specific drivers (haproxy, F5, A10, etc), gradually migrating to Octavia when time will come.

Heat already provides that, though it uses neutron lbaas api extenstions and not the octavia API (you've to set the service provider for lbaas config ex. service_provider = LOADBALANCERV2:Octavia:neutron_lbaas.drivers.octavia.driver.OctaviaDriver:default).

We would probably not like to have the logic in the resources to call two different api endpoints based on the 'provider' choice in resource properties and then provide more functionality for the ones using 'octavia'.

[Volodymyr Litovka (doka.ua)]

Hi,

On 12/13/17 11:03 AM, Rabi Mishra wrote:
> I would suggest we move this to a ML discussion for everyone to
> participate.

Answered in openstack-dev.

Thanks.

--
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison

[Thomas Herve (therve)]

We should just add new resources. I'll be more clear what's going on, and will allow easy addition to support new features from Octavia.

[Volodymyr Litovka (doka.ua)]

Hi colleagues,

thanks for taking care about this issue!

Just for resource planning - milestone 'Openstack Heat next' means 'sometime in future' or something specific?

Thank you.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529474

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529475

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529476

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529477

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529478

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/529479

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/530882

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/530883

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529474
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=d0787393ce1ba073a317c3dccff8251068a707ac
Submitter: Zuul
Branch: master

commit d0787393ce1ba073a317c3dccff8251068a707ac
Author: rabi <email address hidden>
Date: Tue Dec 12 12:40:49 2017 +0530

    Add octavia client plugin and constraints

    Change-Id: I0fa34d19781c49ede8743defe72bbdc63f375896
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/532744

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529475
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=3956746725ec1c97753187296f1da6f804d28dbe
Submitter: Zuul
Branch: master

commit 3956746725ec1c97753187296f1da6f804d28dbe
Author: rabi <email address hidden>
Date: Tue Dec 19 09:28:22 2017 +0530

    Add octavia PoolMember Resource

    Change-Id: I29a66c016714270faf1f2e8d98f1d496e7704e57
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529476
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=512935af781b3d68c6689ab78bb15abe85c9fd36
Submitter: Zuul
Branch: master

commit 512935af781b3d68c6689ab78bb15abe85c9fd36
Author: rabi <email address hidden>
Date: Tue Dec 19 10:30:51 2017 +0530

    Add octavia Pool Resource

    Change-Id: Iaaa18636d799148728d0858c3bd3bab877d23079
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529477
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=6a3a39a7670c47bbbeb187f389b79ebe02ccc421
Submitter: Zuul
Branch: master

commit 6a3a39a7670c47bbbeb187f389b79ebe02ccc421
Author: rabi <email address hidden>
Date: Tue Dec 19 14:24:30 2017 +0530

    Add octavia Listener Resource

    Change-Id: I55e7682f6d2cb7f42c683ee4a6abb7c2106a3d86
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529478
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=d84798b93a8f63b3374bf0c9375601a6a9cbd794
Submitter: Zuul
Branch: master

commit d84798b93a8f63b3374bf0c9375601a6a9cbd794
Author: rabi <email address hidden>
Date: Tue Dec 19 14:26:24 2017 +0530

    Add octavia Loadbalancer Resource

    Change-Id: I05f105d290696e1dedc5c9135a5ee36a1271a7ad
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/529479
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=eb05cacdb651e7d526d5a0a9106ee55f3a9b49b7
Submitter: Zuul
Branch: master

commit eb05cacdb651e7d526d5a0a9106ee55f3a9b49b7
Author: rabi <email address hidden>
Date: Tue Dec 19 14:26:46 2017 +0530

    Add octavia HealthMonitor Resource

    Change-Id: I1e779c4c257f0a57c2f6fdafa5f6fabb72a4b967
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/532744
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=a56e1aaf4e48f36d9b1cf49425f757a9915f7ca3
Submitter: Zuul
Branch: master

commit a56e1aaf4e48f36d9b1cf49425f757a9915f7ca3
Author: rabi <email address hidden>
Date: Thu Jan 11 11:04:34 2018 +0530

    Allow octavia specific client config options

    Allows for using specific endpoint_type for octavia service.
    This was missed in d0787393ce1ba073a317c3dccff8251068a707ac

    Change-Id: I9efc1ba306d11b40f7ca4d609a9282c64debdca3
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/530882
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=cab4258cc80ea79c34096f94318f88a9a6a90210
Submitter: Zuul
Branch: master

commit cab4258cc80ea79c34096f94318f88a9a6a90210
Author: rabi <email address hidden>
Date: Wed Jan 3 11:51:22 2018 +0530

    Add octavia L7Policy Resource

    Change-Id: I05a5d97633ebbe8473c0f582061fa6b1ca6a1358
    Partial-Bug: #1737567

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/530883
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=946c576026f1ecaaed81ec74e44d94b24f9b6fbf
Submitter: Zuul
Branch: master

commit 946c576026f1ecaaed81ec74e44d94b24f9b6fbf
Author: rabi <email address hidden>
Date: Wed Jan 3 14:22:17 2018 +0530

    Add octavia L7Rule Resource

    This also adds releasenotes for the newly added resources.

    Change-Id: Ibaa618d14ea06f6ed2b2fc81d8d3ef270ee16a36
    Closes-bug: #1737567

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/heat 10.0.0.0b3 development milestone.

[Volodymyr Litovka (doka.ua)]

Long live Heat team! Thank you, Rabi, thanks the whole team for an impressive responsiveness, unbelievable performance and, finally, for great product.