No way to override roles_data.yaml

[Steven Hardy]

There's a usability bug with tripleoclient when trying to use custom-roles, because the only way to modify the roles_data.yaml file is to copy the entire templates tree and upload that tree to the plan, e.g

cp -r /usr/share/openstack-tripleo-heat-templates/ tht
vim tht/roles_data.yaml
openstack overcloud deploy --templates tht

This works, but it's inconvenient for folks who want to treat the packaged templates as more of a "black box" vis maintaining (and rebasing) a local copy.

As an alternative, we should enable a similar flow to the -e interface for environments, e.g

openstack overcloud deploy --templates -r custom_roles_data.yaml

I guess a similar interface will be needed when overriding the roles in the default stored plan.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/378740

[Steven Hardy (shardy)]

Ok so after some discussion with Marius and others, there's a few pieces to fix here:

1. Add a roles override (completely override the roles_data file)

2. Provide some way to merge in (e.g append, or redefine when role names collide) user provided roles with those defined by the default roles_data.yaml (so e.g vendor specific roles don't need to care about the default OpenStack services lists)

We need to enable this for both the overcloud deploy and overcloud plan create paths.

I've posted a patch which addresses the first use-case, only for overcloud deploy - this may have to be enough for the Newton GA, then we can circle back and fix the remaining cases afterwards.

[Saravanan KR (skramaja)]

> 2. Provide some way to merge in (e.g append, or redefine when role
> names collide) user provided roles with those defined by the default

Why do we need to merge role definitions if any issues/duplicates/collision in roles_data.yaml? Validate role definitions and stop the deploy should be the way, is what i feel. Please correct me if I am missing something.

[Steven Hardy (shardy)]

@skramaja: the use case is when you want to deploy some extra role, but without having to constantly rebase a custom roles file (e.g when the packaged roles_data.yaml changes, such as to add, remove or rename existing OpenStack Services.

Perhaps "merge" is the wrong term - I mean if say you pass roles-extra.yaml, and it contains a "Controller" role and a "ServiceApi" role (where the latter is running the openstack API services, which are no longer running on the Controller role), we should accept the roles-extra.yaml definition of "Controller", overwriting the one from the default roles_data.yaml (but all other non colliding roles in roles_data.yaml should remain).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/382224

[OpenStack Infra (hudson-openstack)]

Change abandoned by Steven Hardy (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/382224

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/378740
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=a6221608bd7641d7eeac6ad3733d37e51989d9f9
Submitter: Jenkins
Branch: master

commit a6221608bd7641d7eeac6ad3733d37e51989d9f9
Author: Steven Hardy <email address hidden>
Date: Fri Sep 23 11:57:10 2016 +0100

    Add optional overcloud deploy roles_data.yaml override

    Allows users to optionally pass -r roles_data.yaml to override the
    default file in the plan.

    Note this is the first step, a similar interface needs to be added
    to the overcloud plan create command but this may require rework
    of the default plan creation workflow (or some way to re-trigger
    the j2 templating after modifying the roles data).

    Change-Id: I5c6e22fe9516c4e2dad4495127d923f971c32f5d
    Partial-Bug: #1626955

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/382224
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=0ccf0fdd1b01f6628835f7a55f939a8f741cae86
Submitter: Jenkins
Branch: stable/newton

commit 0ccf0fdd1b01f6628835f7a55f939a8f741cae86
Author: Steven Hardy <email address hidden>
Date: Fri Sep 23 11:57:10 2016 +0100

    Add optional overcloud deploy roles_data.yaml override

    Allows users to optionally pass -r roles_data.yaml to override the
    default file in the plan.

    Note this is the first step, a similar interface needs to be added
    to the overcloud plan create command but this may require rework
    of the default plan creation workflow (or some way to re-trigger
    the j2 templating after modifying the roles data).

    Change-Id: I5c6e22fe9516c4e2dad4495127d923f971c32f5d
    Partial-Bug: #1626955
    (cherry picked from commit a6221608bd7641d7eeac6ad3733d37e51989d9f9)

[Jiri Tomasek (jtomasek)]

In terms of enabling user defining custom role via GUI we should possibly keep the roles_data.yaml content in mistral environment. Maybe we could consolidate this file into a plan.yaml which would serve to populate mistral environment when plan is created. The file itself would not be kept in the plan.

We will need this mechanism anyway to be able to suport import/export of the plan (including the mistral environment)

How about providing ability to assign services to a roles, it is preferred to be done via changing roles_data.yaml or setting certain parameter which holds the list of services for a role?

[Steven Hardy (shardy)]

I'm going to mark this one fix released, then we can raise a follow-up bug if the use-case described in comment #2 and comment #4 proves to be a priority (e.g merging into the default roles_data vs requiring the whole file to be defined). The feedback I've got so far is that what we landed is working OK and may therefore be enough.