octavia-api does not support ipv6

[Xing Zhang]

ERROR log for octavia api when deploy octavia with IPv6:

2020-04-22 18:17:45.397 8 INFO octavia.cmd.api [-] Starting API server on 2001:db8::1f:9876
2020-04-22 18:17:45.398 8 CRITICAL octavia [-] Unhandled error: gaierror: [Errno -9] Address family for hostname not supported
2020-04-22 18:17:45.398 8 ERROR octavia Traceback (most recent call last):
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/bin/octavia-api", line 10, in <module>
2020-04-22 18:17:45.398 8 ERROR octavia sys.exit(main())
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib/python2.7/site-packages/octavia/cmd/api.py", line 44, in main
2020-04-22 18:17:45.398 8 ERROR octavia srv = simple_server.make_server(host, port, app)
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 144, in make_server
2020-04-22 18:17:45.398 8 ERROR octavia server = server_class((host, port), handler_class)
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
2020-04-22 18:17:45.398 8 ERROR octavia self.server_bind()
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind
2020-04-22 18:17:45.398 8 ERROR octavia HTTPServer.server_bind(self)
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
2020-04-22 18:17:45.398 8 ERROR octavia SocketServer.TCPServer.server_bind(self)
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
2020-04-22 18:17:45.398 8 ERROR octavia self.socket.bind(self.server_address)
2020-04-22 18:17:45.398 8 ERROR octavia File "/usr/lib64/python2.7/socket.py", line 224, in meth
2020-04-22 18:17:45.398 8 ERROR octavia return getattr(self._sock,name)(*args)
2020-04-22 18:17:45.398 8 ERROR octavia gaierror: [Errno -9] Address family for hostname not supported
2020-04-22 18:17:45.398 8 ERROR octavia

octavia-api in kolla-ansible still using octavia-api and octavia-api start by using simple_server:
https://github.com/openstack/kolla-ansible/blob/ca9e6f1a4e733c85b5d9fa9f352563bd7ebdbcd5/ansible/roles/octavia/templates/octavia-api.json.j2#L2
https://github.com/openstack/octavia/blob/9d50c7918fd55559567463d68d2fcd314471130a/octavia/cmd/api.py#L35-L44

We should switch octavia-api to wsgi running under apache, this also support tls and IPv6.
https://blueprints.launchpad.net/kolla-ansible/+spec/add-ssl-internal-network
https://opendev.org/openstack/octavia/commit/a669d7e4f18b5b7d089af0c7137abc30d73e5de3

[Xing Zhang (xingzhang)]

not only octavia, other services also have same issues.

[Xing Zhang (xingzhang)]

Related Bug:
https://bugs.launchpad.net/tripleo/+bug/1815811

[Piotr Misiak (pmisiak)]

I've deployed Ussuri release with a basic services and IPv6 in control plane.
Besides OpenStack core I enabled a following services:

enable_barbican: "yes"
enable_cinder: "yes"
enable_cinder_backup: "yes"
enable_octavia: "yes"

I also enabled OVN under Neutron.

Octavia API is the only one service not starting due to IPv6 incompatibility.

[Michael Johnson (johnsom)]

The Octavia API is a WSGI application and should never be run via the simple_server based "octavia-api" script in a production deployment. The python simple_server will not scale to production workloads.

We have a warning log message in the code for this: https://github.com/openstack/octavia/blob/master/octavia/cmd/api.py#L44

Octavia uses the PBR WSGI hook[1] to deploy a WSGI stub file on deployment called octavia-wsgi that can be used with most WSGI frameworks. Others can hook directly to the setup_app.

Like the other OpenStack projects that support WSGI, we highly recommend running it under a WSGI framework like uWSGI or gunicorn.

Devstack and the upstream zuul gates run Octavia this way:
https://zuul.opendev.org/t/openstack/build/e21631c64801494694e812fa537bede8/log/controller/logs/etc/octavia/octavia-uwsgi.ini

On a side note, Octavia API has supported IPv6 since the initial Liberty release as far as I can remember. Octavia API has always been a WSGI app[2].

[1] https://github.com/openstack/octavia/blob/master/setup.cfg#L43
[2] https://github.com/openstack/octavia/blob/liberty-eol/octavia/api/app.py#L27

[Mark Goddard (mgoddard)]

Octavia now running under Apache mod_wsgi: https://review.opendev.org/c/openstack/kolla-ansible/+/779893

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/786665

[Will Szumski (willjs)]

Would be nice to cherry-pick this back to earlier releases as the simple http server is pretty flaky. I had a stab with stein:

https://review.opendev.org/c/openstack/kolla-ansible/+/786665

The tricky piece is that the original change depended on:

https://review.opendev.org/c/openstack/kolla/+/779892

Which makes it not possible to deploy a new image with an older version of kolla-ansible. I tried manually overriding the user in the kolla-ansible side so that we can use the existing support in the image. Any thoughts?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/786669

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Will Szumski <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/786665
Reason: Abandoned in favour of https://review.opendev.org/c/openstack/kolla-ansible/+/786669 as I messed up the change-id

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 12.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Michal Nasiadka <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/786669
Reason: stable/stein is going EOL