manila has http endpoints despite ssl_* options being configured

[David Coronel]

When configuring manila with the ssl_ca, ssl_cert and ssl_key charm options, the openstack endpoints for manila still end up using http instead of https:

$ openstack endpoint list | grep http:
| 0b0e6005b1ee4b89ab776642845d9d8c | <region> | manilav2 | sharev2 | True | admin | http://manila-i.<domain>.com:8786/v2/%(tenant_id)s
| 0db6b54e24ae474a936b7adb53ca9d40 | <region> | manila | share | True | public | http://manila.<domain>.com:8786/v1/%(tenant_id)s
| 197e7c9955434d16a8bd6f48b37ca21e | <region> | manila | share | True | internal | http://manila-i.<domain>.com:8786/v1/%(tenant_id)s
| 29892517f2c64ac9b603713dbfc87332 | <region> | manilav2 | sharev2 | True | internal | http://manila-i.<domain>.com:8786/v2/%(tenant_id)s
| 37b2001874b7404ab7fe40079f8c2d5a | <region> | manila | share | True | admin | http://manila-i.<domain>.com:8786/v1/%(tenant_id)s
| 8b9b176894be4839a1ded238cb9a9b5e | <region> | manilav2 | sharev2 | True | public | http://manila.<domain>.com:8786/v2/%(tenant_id)s

The endpoints can be temporarily fixed by using the "openstack endpoint set --url" command, but the endpoints revert back to http after some time.

This is an OpenStack from the bionic-train cloud archive.

manila charm rev 92
manila packages version 1:9.0.0-0ubuntu1~cloud0

[David Coronel (davecore)]

subscribed ~field-critical

[David Coronel (davecore)]

When the endpoints revert back to http, they actually are completely new endpoints:

before:

62bf38b199154b84b5a5ee42996fe429
b1a57edd18f54bb1b074b179dcc42f14
bd72a95d68a84cefb5e842f40ce8504f
cfb92afc922a41a898eed06a787b600a
e7e305c735884078b72c264c708c5aa8
e829f5cc495d4a29aafb30dc3efa40aa

After the revert:

0b0e6005b1ee4b89ab776642845d9d8c
0db6b54e24ae474a936b7adb53ca9d40
197e7c9955434d16a8bd6f48b37ca21e
29892517f2c64ac9b603713dbfc87332
37b2001874b7404ab7fe40079f8c2d5a
8b9b176894be4839a1ded238cb9a9b5e

[David Coronel (davecore)]

The reverts to http happened when we did updates on keystone.

[Billy Olsen (billy-olsen)]

What charm revision/branch are you deploying from?

I believe that this should be fixed in the openstack-charmers-next charms with https://github.com/openstack/charms.openstack/commit/69eb753b02adb5d5a589fdded49094df38e086d3 change.

[Billy Olsen (billy-olsen)]

Actually, I'm still seeing this on a deploy from master. A work-around is to set the relation information manually, i.e.:

$ juju run --unit manila/leader 'relation-ids identity-service'
identity-service:49

$juju run --unit manila/leader 'relation-set -r identity-service:49 v1_public_url="https://10.5.150.1:8786/v1/%(tenant_id)s" v1_internal_url="https://10.5.150.1:8786/v1/%(tenant_id)s" v1_public_url="https://10.5.150.1:8786/v1/%(tenant_id)s"'

$ juju run --unit manila/leader 'relation-set -r identity-service:49 v2_admin_url="https://10.5.150.1:8786/v2/%(tenant_id)s" v2_internal_url="https://10.5.150.1:8786/v2/%(tenant_id)s" v2_public_url="https://10.5.150.1:8786/v2/%(tenant_id)s"'

[Billy Olsen (billy-olsen)]

grr, typo in previous command set (specified v1_public_url twice) - you can do it in one shot:

$ juju run --unit manila/leader 'relation-ids identity-service'
identity-service:49

$ juju run --unit manila/leader 'relation-set -r identity-service:49 v2_admin_url="https://10.5.150.1:8786/v2/%(tenant_id)s" v2_internal_url="https://10.5.150.1:8786/v2/%(tenant_id)s" v2_public_url="https://10.5.150.1:8786/v2/%(tenant_id)s" v1_public_url="https://10.5.150.1:8786/v1/%(tenant_id)s" v1_internal_url="https://10.5.150.1:8786/v1/%(tenant_id)s" v1_admin_url="https://10.5.150.1:8786/v1/%(tenant_id)s"'

[Billy Olsen (billy-olsen)]

Looking at the code a bit more, I think this patch might solve it- not submitting as its a bit late and I need to look at it more before submitting:

diff --git a/src/reactive/manila_handlers.py b/src/reactive/manila_handlers.py
index daf6744..c62c732 100644
--- a/src/reactive/manila_handlers.py
+++ b/src/reactive/manila_handlers.py
@@ -114,6 +114,10 @@ def render_stuff(*args):
     was set.
     """
     with charms_openstack.charm.provide_charm_instance() as manila_charm:
+ tls = relations.endpoint_from_flag('certificates.available')
+ manila_charm.configure_tls(certificates_interface=tls)
+ keystone = relations.endpoint_from_flag('identity-service.available')
+ manila_charm.register_endpoints(keystone)
         manila_charm.render_with_interfaces(args)
         manila_charm.assess_status()
         charms.reactive.set_state('manila.config.rendered')

Essentially, the charm has a default config_changed but doesn't render the tls configuration, nor does it advertise the updated endpoints to keystone. I might be missing it in my perusal of the code, but the above works for me.

[David Coronel (davecore)]

I applied the workaround for now. We can test a patch once it's ready. Thanks.

[Billy Olsen (billy-olsen)]

With the work around applied, we can drop this to a field-high

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/760233

[Derek Robertson (rober546)]

Does anyone know when this patch is likely to make it into the openstack-charmers-next on charm store please ?

[Alex Kavanagh (ajkavanagh)]

> Does anyone know when this patch is likely to make it into the openstack-charmers-next on charm store please ?

The review is passing on gerrit and (assuming it passes it's recheck-full) should be good to go early next week.

[Liam Young (gnuoy)]

Looking at the functional test failure all the manilla api units were shutdown when the test attempted to list endpoints. I have raised https://github.com/openstack-charmers/zaza-openstack-tests/pull/481 which I think extends the existing band-aid to fix the tests.

[Liam Young (gnuoy)]

Hmm, I was wrong its the manila-share service that may be delayed starting not the manila-api service the problem in the tests is probably causes by something else.