When using SSL service randomly fails

[Liam Young]

When charm has been setup with SSL enabled, connections to external endpoints randomly fail. The service log contains messages like:

Caught error: SSL exception connecting to <REMOTE THING>: [Errno 2] No such file or directory

This appears to be because when the update-status hook runs it regenerates the certificates. During the regeneration attempts to connect external endpoints will fail because the certs are absent.

# grep -E "Caught.*No such file" /var/log/nova/nova-api-os-compute.log | awk '{print $2}'
19:54:12.665
02:51:51.969
02:51:52.294
05:00:40.807
05:21:51.992
05:21:52.296
07:41:03.963
08:13:05.970
08:23:47.070
08:50:40.425
09:06:47.632
09:06:48.707
09:22:51.706
09:44:25.982
09:55:26.594
09:55:29.301
10:07:14.554
10:12:35.789
root@juju-trusty-machine-12:/var/lib/juju/agents/unit-nova-cloud-controller-0/charm# grep "Updating certificates" /var/log/juju/unit-nova-cloud-controller-0.log | tail -10
2016-07-21 09:38:54 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 09:44:17 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 09:49:40 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 09:55:21 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:01:01 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:07:10 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:12:33 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:17:56 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:23:19 INFO update-status Updating certificates in /etc/ssl/certs...
2016-07-21 10:28:40 INFO update-status Updating certificates in /etc/ssl/certs...

[Liam Young (gnuoy)]

Fixes committed to charms here:

https://review.openstack.org/#/q/topic:chsync1607