"unsupported operand type" error during unit teardown caused by stale certificates.server.cert.available flag

[Paul Goins]

This actually is a bug in layer:tls-client, but as charm-kubernetes-worker uses it, it is affected. Likely other charms are affected as well.

Link to the layer:tls-client bug: https://github.com/juju-solutions/layer-tls-client/issues/22

Quick summary: when removing a unit, depending on the order that peer relations are removed and which remote unit holds the server_cert info, a unit may no longer have server certificate info (e.g. <unit>.server.cert) provided via its certificates relation. Unfortunately, handlers may fire in an order where availability flags may not get properly updated before running hooks that depend on certs being available, thus resulting in errors such as this:

  File "/var/lib/juju/agents/unit-kubernetes-worker-26/charm/reactive/tls_client.py", line 51, in store_server
    server_cert = server_cert + '\n' + chain
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

See the GitHub ticket for additional context as well as a potential workaround, if needed.

[George Kraft (cynerva)]

I believe the correct place to fix this is in interface-tls-certificates. The joined method[1] should be replaced with an implementation of manage_flags[2], which is guaranteed to be called before reactive handlers run. That would ensure that certificates.server.cert.available is cleared before layer-tls-client handlers have a chance to run.

[1]: https://github.com/charmed-kubernetes/interface-tls-certificates/blob/d9850016d930a6d507b9fd45e2598d327922b140/requires.py#L79-L80
[2]: https://github.com/juju-solutions/charms.reactive/blob/f4f5ac61713a5544c3c522fa5873157a383241d7/charms/reactive/endpoints.py#L253