ssl not served when https-service-endpoints are enabled

[Billy Olsen]

In a MAAS deployment which is serving providing dns for the nodes the service endpoints are registered with keystone as the fqdn of the node name. The apache proxy used to configure the SSL will do so for a virtual host which is named which is not used by the time the request hits the server since the clients etc will turn the hostname into an IP.

For example, in my virtual MAAS cluster I have a cinder node which is named cinder.wolsen.local. This will cause the charm to advertise its service endpoints to keystone as https://cinder.wolsen.local:8776/v2 and the apache https configuration to look as follows:

Listen 8776
<VirtualHost cinder.wolsen.local:8776>
    ServerName cinder.wolsen.local
    SSLEngine on
    ...
    PRoxyPreserveHost on
</VirtualHost>

The problem is that this configuration binds the virtualhost for cinder.wolsen.local which isn't served because the requests come in for the IP address.

Attempts to communicate with the service using (for example) the cinder list command fails with the following:

ubuntu@horizon:~$ cinder list
ERROR: Unable to establish connection: [Errno 1] _ssl.c:510: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
ubuntu@horizon:~$

[Billy Olsen (billy-olsen)]

I was able to recreate the bug using a MAAS installation, but upon further investigation it appears that it is caused by the network splits and not MAAS environment specifically.

In this case, there are no os-xxx-networks defined and there are only single units deployed of each service. This result is a configuration which has an invalid https configuration.

[Billy Olsen (billy-olsen)]

One work-around is to specify the os-public-network as the current network which it resides upon.

[Edward Hope-Morley (hopem)]

Billy. I think I have hit this issue and know what is causing it. Assuming that you are using a single unit of each service affected i.e. keystone, cinder, glance etc, you will be hitting the following:

Endpoint will configure apache to listen on the same port used for haproxy. Result is that apache2 service won't start because it can't bind to that port and so ssl fails. I currently see this for all ssl-enabled services. The problem goes away if you use > 1 unit due to the logic in http://bazaar.launchpad.net/~charm-helpers/charm-helpers/devel/view/head:/charmhelpers/contrib/hahelpers/cluster.py#L189

So, i'll let you confirm this is the cause of your issue before taking this LP but I will fix the above issue in any case since it currently causes non-HA ssl to fail for all services.

[Billy Olsen (billy-olsen)]

Ed, I think the problem is different in the stable version of the charms. In the next branches, this is possibly an issue - however in the cases I've seen this issue encountered there is no issue in binding to the ssl port, rather the Apache virtual server is configured to listen on a public endpoint. This is due to not having any of the os-xxx-networks defined - the call on http://bazaar.launchpad.net/~charm-helpers/charm-helpers/devel/view/head:/charmhelpers/contrib/openstack/context.py#L626 will simply return the default private address for the unit, which happens to be the hostname and apache does not start the ssl listening for this particular port.

[James Page (james-page)]

Billy - what entry is in the service catalog compared to the configured vhost in apache?