overcloud_containers.yaml should be template driven, not static

[Steve Baker]

Currently container based CI jobs are implementing their own logic to
transform overcloud_containers.yaml to a file that reflects the
container images and registry required to run their job [1] unless the
job requires the exact images which the static
overcloud_containers.yaml provides[2].

overcloud_containers.yaml should be template driven, and a command implemented which meets the needs of downstream for declaring what images are used.

This command can also generate a heat environment file with the parameters containing image values.

[1] https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/tasks/main.yml#L39-L75
[2] https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L18

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/475942

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/475958

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/475975

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/471537
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=dc0b057f6e2fdf17ff84f51e5a5af7897226bf12
Submitter: Jenkins
Branch: master

commit dc0b057f6e2fdf17ff84f51e5a5af7897226bf12
Author: Steve Baker <email address hidden>
Date: Wed Jun 7 09:14:52 2017 +1200

    Make overcloud_containers.yaml template driven

    Currently container based CI jobs are implementing their own logic to
    transform overcloud_containers.yaml to a file that reflects the
    container images and registry required to run their job [1] unless the
    job requires the exact images which the static
    overcloud_containers.yaml provides[2].

    This change adds a template overcloud_containers.yaml.j2 which will
    exist in parallel to overcloud_containers.yaml until all downstream
    users of overcloud_containers.yaml switch to a templated approach.

    This change is the first step to write commands which allow
    overcloud_containers.yaml to be built for a specific enviroment,
    including version based tags and reducing the image list to those
    needed by the deployment.

    The test test_container_images_yaml_in_sync will ensure that any
    changes made to overcloud_containers.yaml won't pass until they're
    made to overcloud_containers.yaml.j2 as well. This will keep them in
    sync until overcloud_containers.yaml is removed.

    [1] https://github.com/rdo-infra/ansible-role-rdo-kolla-build/blob/master/tasks/main.yml#L39-L75
    [2] https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L18

    Change-Id: I4fe18565362f9f308dd9957aeb9bc0f7498590a6
    Partial-Bug: #1696598

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/477752

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/477752
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=7d83db14f2a53a68ae0b5f411fa3197146ef0411
Submitter: Jenkins
Branch: master

commit 7d83db14f2a53a68ae0b5f411fa3197146ef0411
Author: Steve Baker <email address hidden>
Date: Tue Jun 27 17:13:20 2017 +1200

    Map container image entries to stack params

    This change contains the data required to generate a heat environment
    file which populates stack parameters with the desired container image
    strings. This file will be generated when "overcloud container image
    prepare" is called.

    Change-Id: Id15f7d95640c47640215edf09e7b99d2fd873251
    Partial-Bug: #1696598

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/475942
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=daf0f04bcfc56fb4516a3c6b13aeb5396f8b44f9
Submitter: Jenkins
Branch: master

commit daf0f04bcfc56fb4516a3c6b13aeb5396f8b44f9
Author: Steve Baker <email address hidden>
Date: Wed Jun 21 08:50:24 2017 +1200

    New command "overcloud container image prepare"

    This new command allows the user to generate their own
    overcloud_containers.yaml file in preperation for calling
    "overcloud container image build" or
    "overcloud container image upload".

    Arguments allow the caller to customise the resulting
    overcloud_containers.yaml file with the aim of not requiring any
    further manual editing for the majority of cases.

    The file generated by the --env-file option adopts the convention
    established in change Ieaedac33f0a25a352ab432cdb00b5c888be4ba27 where
    the DockerNamespace parameter is not used and image names are fully
    qualified.

    Partial-Bug: #1696598
    Change-Id: I6af1828ea2b26f3c6295981fe40fe825d8ccef37

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/475958
Committed: https://git.openstack.org/cgit/openstack/tripleo-docs/commit/?id=adf1d32d134944fb006eaa0844d4d9f50d5b50e2
Submitter: Jenkins
Branch: master

commit adf1d32d134944fb006eaa0844d4d9f50d5b50e2
Author: Steve Baker <email address hidden>
Date: Wed Jun 21 11:47:53 2017 +1200

    Document the containers prepare command

    Change-Id: Icf040fd41de0ff2c82520e8d22834414ff7eccd9
    Partial-Bug: #1696598

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/475975
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=90e703768326622eab1cf8dfa80daddccc3f88c8
Submitter: Jenkins
Branch: master

commit 90e703768326622eab1cf8dfa80daddccc3f88c8
Author: Steve Baker <email address hidden>
Date: Wed Jun 21 13:24:41 2017 +1200

    Use prepare command for overcloud_containers.yaml

    Most deployments will not be able to use the packaged
    overcloud_containers.yaml unmodified, so the prepare command will
    allow local modified files to be created.

    Once nothing in CI consumes the packaged overcloud_containers.yaml, it
    will be deleted.

    This change also used the --env-file option to generate
    container-default-parameters.yaml instead of the current templated
    approach. There is about to be some disruption to how the namespace is
    represented in the image parameters, so using the prepare command will
    help mitigate that.

    Change-Id: I6a5796428c224584359b2a434d7fc80878d5f038
    Partial-Bug: #1696598

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/487613
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=2597449b17dd72fe3173286ec04d13b6fe2bd830
Submitter: Jenkins
Branch: master

commit 2597449b17dd72fe3173286ec04d13b6fe2bd830
Author: Steve Baker <email address hidden>
Date: Thu Jul 27 10:23:39 2017 +1200

    Delete docker-centos-tripleoupstream.yaml

    This file is generated and needs to be manually maintained. It
    would be better for users who want to deploy latest directly from
    docker hub to generate it locally by running:

        openstack overcloud container image prepare \
          --namespace tripleoupstream \
          --tag latest \
          --env-file docker-centos-tripleoupstream.yaml

    The documentation and CI are being updated to use prepare.

    Change-Id: I86503f1076459ae9d84a34e649a6097cba10fa3c
    Closes-Bug: #1696598

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.0.0rc1 release candidate.