apache2 is not reloaded when updating SSL certificate

[Tytus Kurek]

It looks like apache2 is not reloaded when updating SSL certificate. This results in API errors, if the hostname in the certificate changes:

$ openstack zone list
Failed to contact the endpoint at https://100.86.0.11:9001 for discovery. Fallback to using that endpoint as the base url.
SSL exception connecting to https://100.86.0.11:9001/zones?: hostname '100.86.0.11' doesn't match 'designate.example.com'

The workaround is to log into designate units and reload apache2 manually.

Attaching log files from all units (the change was done around 11:30 UTC).

[Tytus Kurek (tkurek)]

Fix proposed to branch: master
Review: https://review.openstack.org/#/c/539962/

[Tytus Kurek (tkurek)]

I've created a pull request for charm-helpers:

https://github.com/juju/charm-helpers/pull/105

[Tytus Kurek (tkurek)]

I've re-created the pull request for charm-helpers:

https://github.com/juju/charm-helpers/pull/109

[Frode Nordahl (fnordahl)]

Digging a bit deeper into this.

I assume the certificate is changed through Keystone Charm. From your description I also assume you use the Keystone ssl_ca, ssl_cert and ssl_key charm configuration options to change the certificate.

The existing classic charms currently handles this by reloading Apache in their config-changed hook when SSL is enabled.

Question:
How does the other reactive OpenStack Charms (aodh, gnocchi, etc) currently behave when a certificate is changed in this manner? Do they require manual restart/reload of Apache too?

If the other reactive OpenStack Charms has the same issue but the classic OpenStack Charms does not I am inclined to ask the question if this should be handled in conjunction with the reactive keystone interface layer instead of in charm-helpers? (https://github.com/openstack/charm-interface-keystone/)

[Tytus Kurek (tkurek)]

Frode,

The issue was originally experienced when updating the certificate for the designate service. I made a debug and noticed that apache2 is indeed not restarted when performing such a change. Then I noticed that "charm-helpers" are missing the same part of the code (I didn't make a debug), so I raised it against "charm-helpers" as well. So, to be honest, I haven't performed any tests with services other than designate. Thus, if the issue is not present in classic charms (which we don't know yet), I am happy to close the pull request I created.

On the other, if the issue is present in reactive charms only, I don't see a reason to fix it in the "charm-interface-keystone" instead of "charms.openstack".

[Frode Nordahl (fnordahl)]

The thing that makes this a bit tricky is that there are two modes of operations here.

Mode 1) Certificates managed by Keystone Charm
In this mode Keystone Charm make related charms restart their apache2 service through peer actions after synchronizing the certificates and keys. I tested a few reactive charms and they act on the peer_action for restart after initial setup at least and that makes me think they will do so in the event of certificate changes as well for this mode.

I initially thought you were addressing this use-case which was why I mentioned the keystone interface layer. Since that part seems to work OK we can look away from that.

Mode 2) Certificates set up through config directly on each individual charm
I guess this is the mode you are addressing with this bug.

We need to take care and add the reload in such a way that it does not lead to multiple reloads of apache2 in reactive charms for Mode 1.

[Tytus Kurek (tkurek)]

Yes, mode 2 is the one I'm addressing.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/539962
Committed: https://git.openstack.org/cgit/openstack/charms.openstack/commit/?id=e2f2d9ea2f61e3f62eef59a9343856b4a900d900
Submitter: Zuul
Branch: master

commit e2f2d9ea2f61e3f62eef59a9343856b4a900d900
Author: Tytus Kurek <email address hidden>
Date: Wed Feb 14 15:08:23 2018 +0100

    Restart apache2 on SSL certificate change

    This patchset implements a logic to restart apache2 on SSL
    certificate change.

    Change-Id: I9904a92feeee64233f3d07c26ca8dd60920feeb0
    Closes-Bug: #1745985