Service accounts duplicated in domains, causes some heat stack actions to fail

[Xav Paice]

On building Gnocchi and Aodh, the user account for the services is created both in 'default' domain plus the 'service_domain', and a project 'services' rather than 'service' (the default).

This causes a few issues, noticed in our case when rolling a heat stack that includes gnocchi based aodh alarms:

openstack stack create --wait -t web.yaml test1
2018-06-11 21:56:01Z [test1]: CREATE_IN_PROGRESS Stack CREATE started
2018-06-11 21:56:02Z [test1.cpu_alarm_low]: CREATE_IN_PROGRESS state changed
2018-06-11 21:56:03Z [test1.router]: CREATE_IN_PROGRESS state changed
2018-06-11 21:56:03Z [test1.router]: CREATE_FAILED StackValidationFailed: resources.router: Property error: Properties.external_gateway_info.network: Unable to find network with name or id 'floating_nonprod'
2018-06-11 21:56:03Z [test1]: CREATE_FAILED Resource CREATE failed: StackValidationFailed: resources.router: Property error: router.Properties.external_gateway_info.network: Unable to find network with name or id 'floating_nonprod'
2018-06-11 21:56:05Z [test1.cpu_alarm_low]: CREATE_FAILED ClientException: resources.cpu_alarm_low: You are not authorized to perform the requested action: identity:list_projects. (HTTP 403) (Request-ID: req-75cd8caf-3544-417e-9baf-5ed45ccc827b) (HTTP 500) (Request-ID: req-26f153d9-efac-4809-a842-96e99fdba78e)
2018-06-11 21:56:05Z [test1]: CREATE_FAILED Resource CREATE failed: ClientException: resources.cpu_alarm_low: You are not authorized to perform the requested action: identity:list_projects. (HTTP 403) (Request-ID: req-75cd8caf-3544-417e-9baf-5ed45ccc827b) (HTTP 500) (Request-ID: req-26f153d9-efac-4

Stack test1 CREATE_FAILED

Refer to https://docs.openstack.org/aodh/latest/configuration/aodh-config-file.html - the gnocchi_external_project_owner option defaults to a project named 'service' which doesn't exist in our default deployments. This should be set in the template:

In aodh.conf:
[api]
gnocchi_external_project_owner = services

[Xav Paice (xavpaice)]

Additionally, we find the following are needed in aodh.conf in order to overcome the lack of specifics with the config where we have a services_domain and 'default' domain.

[service_credentials]
project_domain_name = service_domain
user_domain_name = service_domain

From another layer, we also have ./templates/parts/section-keystone-authtoken which needs:

[keystone_authtoken]
project_domain_name = service_domain
user_domain_name = service_domain

Currently that section is set to 'default'.

If we didn't add all the services to both the 'default' and the 'services_domain' I wonder if we'd have had such a problem?

[Xav Paice (xavpaice)]

gnocchi.conf also needs editing:

* do sed -i 's/default/service_domain/g' /var/lib/juju/agents/unit-gnocchi-0/charm/templates/parts/section-keystone-authtoken
* sed -i 's/default/service_domain/g' /etc/gnocchi/gnocchi.conf

Further, need to confirm that the password for the accounts is set in the correct domain.

[Xav Paice (xavpaice)]

https://review.openstack.org/#/c/574571/ and https://review.openstack.org/#/c/574920/ should fix this

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/18.05
Review: https://review.openstack.org/575261

[OpenStack Infra (hudson-openstack)]

Change abandoned by Xav Paice (<email address hidden>) on branch: master
Review: https://review.openstack.org/574571
Reason: revised in another change

[OpenStack Infra (hudson-openstack)]

Change abandoned by Xav Paice (<email address hidden>) on branch: stable/18.05
Review: https://review.openstack.org/575261
Reason: wrong branch

[OpenStack Infra (hudson-openstack)]

Change abandoned by Xav Paice (<email address hidden>) on branch: master
Review: https://review.openstack.org/575263
Reason: see https://review.openstack.org/#/c/578289/2/src/templates/aodh.conf

[Drew Freiberger (afreiberger)]

It appears that charm-aodh got built before interface:keystone in layer-openstack was up-to-date with the patch. I've asked @shaner to re-build aodh charm. Might want to check any other related charms that the hooks/relations/keystone/requires.py includes service_domain in the auto_accessors list.

[Xav Paice (xavpaice)]

cs:aodh got a rebuild, the fix is in cs:aodh-17