SSL cert and key options do not work with multiple VIPs

[Darryl Weaver]

We tried enabling the SSL cert and SSL key options in all the Openstack charms.
However, when using multiple networks and multiple VIPs the SSL options generate a certificate per IP address from the management network.
So, you end up with the following files:
$ find /etc/apache2/ssl/
/etc/apache2/ssl/
/etc/apache2/ssl/keystone
/etc/apache2/ssl/keystone/cert_10.5.0.114
/etc/apache2/ssl/keystone/key_10.5.0.205
/etc/apache2/ssl/keystone/key_10.5.0.114
/etc/apache2/ssl/keystone/cert_10.5.0.205

Where 10.5.0.0/24 is the management network and 10.5.0.114 is the DHCP IP and 10.5.0.205 is the VIP on the same network.
But there is also a public IP on 31.28.88.0/24 and a Public VIP on 31.28.88.12 which have no SSL cert created, but the configuration includes it, so apache2 refuses to restart with the error:
AH00526: Syntax error on line 14 of /etc/apache2/sites-enabled/openstack_https_frontend.conf:
SSLCertificateFile: file '/etc/apache2/ssl/keystone/cert_31.28.88.12' does not exist or is empty
Action 'configtest' failed.

Line 14 is: SSLCertificateFile /etc/apache2/ssl/keystone/cert_31.28.88.12

Therefore enabling SSL on any of the Openstack Charms with multiple NICs with a VIP for HA is currently broken.

[James Page (james-page)]

Groking the code:

    def configure_cert(self, cn=None):
        ssl_dir = os.path.join('/etc/apache2/ssl/', self.service_namespace)
        mkdir(path=ssl_dir)
        cert, key = get_cert(cn)
        if cn:
            cert_filename = 'cert_{}'.format(cn)
            key_filename = 'key_{}'.format(cn)
        else:
            cert_filename = 'cert'
            key_filename = 'key'

        write_file(path=os.path.join(ssl_dir, cert_filename),
                   content=b64decode(cert))
        write_file(path=os.path.join(ssl_dir, key_filename),
                   content=b64decode(key))

A cert and key should be written out for all required CN's - if ssl_cert and ssl_key configuration options are used, the same cert and key will be used for all of them - so you'd need to add SAN's to the cert to cover all endpoint addresses.

I think that this should work - as this is quite an old bug it would be good to confirm

[James Page (james-page)]

Hmm more grokking:

        cns = self.canonical_names()
        if cns:
            for cn in cns:
                self.configure_cert(cn)
        else:
            # Expect cert/key provided in config (currently assumed that ca
            # uses ip for cn)
            cn = resolve_address(endpoint_type=INTERNAL)
            self.configure_cert(cn)

for the keystone charm itself, I suspect that we're only configuring the internal endpoint cert and key which looks like the issue in your original bug report.

[James Page (james-page)]

Actually I stand corrected - I think this should work:

     def canonical_names(self):
        addresses = self.get_network_addresses()
        addrs = []
        for address, endpoint in addresses:
            addrs.append(endpoint)

        return list(set(addrs))

the keystone charm overrides the default behaviour for canonical_names in the ApacheSSLContext to use the addresses provided by get_network_addresses, which should include all configuration based network address mappings for internal, admin and public endpoints.

[James Page (james-page)]

Daryl

Appreciate this bug has been around for a while - can you confirm whether this is still a problem for you? Leaving the importance as High for now but marking 'Incomplete'.

Cheers

James

[Peter Sabaini (peter-sabaini)]

I'm seeing this too on n-c-c I think. We have 2 n-c-c vip's defined. When setting ssl_* options I get
/etc/apache2/ssl/nova/{cert,key}_<vip1> configured, but

# grep SSLCertificateFile /etc/apache2/sites-available/openstack_https_frontend.conf
    SSLCertificateFile /etc/apache2/ssl/nova/cert_<vip1>
    SSLCertificateFile /etc/apache2/ssl/nova/cert_<vip2>

n-c-c charm 16.07

[Peter Sabaini (peter-sabaini)]

FWIW, I've seen the behaviour above on the n-c-c service -- but the ks service with same charm rel. and also 2 VIPs got 2x cert files deployed, /etc/apache2/ssl/keystone/cert_<vip2> and _<vip2> with accompanying keyfiles.

[Alvaro Uria (aluria)]

To add more info about n-c-c SSL config:
"""
01a1322e961cc2afb4a0622a5463688eb0cc14f4 /etc/apache2/ssl/nova/cert_VIP1
01a1322e961cc2afb4a0622a5463688eb0cc14f4 /etc/apache2/ssl/nova/cert_VIP2
01a1322e961cc2afb4a0622a5463688eb0cc14f4 /etc/apache2/ssl/nova/cert_OS-INTERNAL-HOSTNAME
01a1322e961cc2afb4a0622a5463688eb0cc14f4 /etc/apache2/ssl/nova/cert_OS-PUBLIC-HOSTNAME
c1627213670e351bd9ad68e79508bf0cfbf8d25f /etc/apache2/ssl/nova/key_VIP1
c1627213670e351bd9ad68e79508bf0cfbf8d25f /etc/apache2/ssl/nova/key_VIP2
c1627213670e351bd9ad68e79508bf0cfbf8d25f /etc/apache2/ssl/nova/key_OS-INTERNAL-HOSTNAME
c1627213670e351bd9ad68e79508bf0cfbf8d25f /etc/apache2/ssl/nova/key_OS-PUBLIC-HOSTNAME
"""

So, the same certificates are configured OK for all VIPs (stable/16.07), but that would only work when using wildcards (ie. *.domain.tld), or if using DNS views for the same CNs (but mapping to internal/public networks). How could we set different certs per internal/public/admin endpoints (CN=internal.domain.tld, CN=public.domain.tld)?

All oscharms use the same charm-helpers code, so behavior is the same on all services.

[Nobuto Murata (nobuto)]

I'm not sure my issue matches this bug, so cross linking just in case:
https://bugs.launchpad.net/charm-helpers/+bug/1711360

[Alvaro Uria (aluria)]

In my case, it also matches Nobuto's (#8) bug, since we have multi-VIP and different hostnames configured on os-{internal,admin}-hostname and os-public-hostname.

The missing file is "cert_<the-2nd-ip>" (and "key_<the-2nd-ip>"). Just seen in swift-proxy (17.02).

[James Page (james-page)]

Looking at the info in bug 1711360 I think this issue was fixed towards the end of last year; the bug has not had a new comment since February, which pre-dates our last two charm releases so marking bug tasks as Fix Released.