prepare command needs to discover the versioned tag from the latest tag

[Steve Baker]

For minor updates of containers to work, tags like 'latest' should never be used. However discovering what current version-based tag to use is a manual process. As a consequence upstream CI and development is exclusively using 'latest' anyway.

This can be solved by having the "openstack overcloud container image prepare" command optionally introspect the registry to substitute the stable (latest) tag to a version based tag.

The RDO/kolla build chain sets the label "rdo_version" to the tag that corresponds to the DLRN ref which that container was built from. For example:

  docker pull trunk.registry.rdoproject.org/tripleo/centos-binary-aodh-api:latest
  docker inspect --format '{{ index .Config.Labels "rdo_version"}}' trunk.registry.rdoproject.org/tripleo/centos-binary-aodh-api:latest

  # 184244eb4ad0a5bd2e5ea7bc68cfbc2251be9d4d_f4d6532a

As long as every downstream image build pipeline sets their own label which states the version-based tag, the prepare command can have a general tag discovery mechanism which works with images from all sources.

I propose that the "openstack overcloud container image prepare" get a --tag-label argument which when specified will do the following:

- introspect the image in the registry corresponding to the specified --tag
- substitute the --tag tag with the value of the --tag-label label
- use this value for any --env-file or --images-file generated files

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/498678

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/498683

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/489837
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=c58ac826df6628629eeddf9e9a5b2f2be5578191
Submitter: Jenkins
Branch: master

commit c58ac826df6628629eeddf9e9a5b2f2be5578191
Author: Steve Baker <email address hidden>
Date: Mon Aug 21 16:58:09 2017 +1200

    Upload remove default for pull_source

    Currently the pull_source defaults to "docker.io". This is unnecessary
    as docker.io is the default registry anyway when no registry is
    specified in the namespace.

    Also having this default prevents fully qualified image paths being
    specified in the imagename. This is desirable now that we use full
    paths in the heat parameters.

    This also allows the prepare command to deprecate the --pull-source
    argument and document having full image paths in the --namespace
    argument.

    Change-Id: I1769c7cf1c058328a8490228bdc4719e8d216ccb
    Partial-Bug: #1708967

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/498678
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=39f5716c041ff3fc9b5b5d68049e8ca474453e1d
Submitter: Jenkins
Branch: master

commit 39f5716c041ff3fc9b5b5d68049e8ca474453e1d
Author: Steve Baker <email address hidden>
Date: Tue Aug 29 12:59:50 2017 +1200

    Discover a versioned container image tag

    This change implements a discover_image_tag method to discover the
    versioned tag for a container image.

    The current approach is to expect a label to be set on the image which
    contains the expected tag value. For example the RDO image building
    pipeline sets a label rdo_version whose value also matches a tag.

    This change will be used by the python-tripleoclient change which
    actually implements the command
    "openstack overcloud container image tag discover".

    Change-Id: I27ea031287604d70032fb5392aecbce313d4b096
    Partial-Bug: #1708967

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/500438

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500438
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=45acbce815d42468288276c694f6597e04b7603d
Submitter: Jenkins
Branch: stable/pike

commit 45acbce815d42468288276c694f6597e04b7603d
Author: Steve Baker <email address hidden>
Date: Mon Aug 21 16:58:09 2017 +1200

    Upload remove default for pull_source

    Currently the pull_source defaults to "docker.io". This is unnecessary
    as docker.io is the default registry anyway when no registry is
    specified in the namespace.

    Also having this default prevents fully qualified image paths being
    specified in the imagename. This is desirable now that we use full
    paths in the heat parameters.

    This also allows the prepare command to deprecate the --pull-source
    argument and document having full image paths in the --namespace
    argument.

    Change-Id: I1769c7cf1c058328a8490228bdc4719e8d216ccb
    Partial-Bug: #1708967
    (cherry picked from commit c58ac826df6628629eeddf9e9a5b2f2be5578191)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/502349

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/502349
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=33d95b3da489f488fa21a50cb4f584042e710eeb
Submitter: Jenkins
Branch: stable/pike

commit 33d95b3da489f488fa21a50cb4f584042e710eeb
Author: Steve Baker <email address hidden>
Date: Tue Aug 29 12:59:50 2017 +1200

    Discover a versioned container image tag

    This change implements a discover_image_tag method to discover the
    versioned tag for a container image.

    The current approach is to expect a label to be set on the image which
    contains the expected tag value. For example the RDO image building
    pipeline sets a label rdo_version whose value also matches a tag.

    This change will be used by the python-tripleoclient change which
    actually implements the command
    "openstack overcloud container image tag discover".

    Change-Id: I27ea031287604d70032fb5392aecbce313d4b096
    Partial-Bug: #1708967
    (cherry picked from commit 39f5716c041ff3fc9b5b5d68049e8ca474453e1d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/498683
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=eca5393ddcced5187b9d360a0b5edb96ca2a534e
Submitter: Jenkins
Branch: master

commit eca5393ddcced5187b9d360a0b5edb96ca2a534e
Author: Steve Baker <email address hidden>
Date: Mon Aug 28 11:20:06 2017 +1200

    Command to discover the versioned tag from latest

    This change implements the command
    "openstack overcloud container image tag discover" which will become
    part of the update and upgrade workflow for deploying new container
    images.

    It is used to discover the version-based tag by inspecting the image
    from a stable tag like current-tripleo-rdo. Stable tags like 'latest'
    or 'pike' can't be used for container updates because something needs
    to change to trigger the new containers being pulled. Without this
    command it would be up to the user to find out what versioned tag to
    specify when calling prepare.

    This was implemented as its own command instead of integrating with
    prepare because there may be multiple image build chains feeding into
    the image list (such as RDO, ceph and eventually others) and each will
    need its own call to discover the versioned tag.

      tag=$(openstack overcloud container image tag discover \
            --image trunk.registry.rdoproject.org/pike/centos-binary-base:current-tripleo-rdo
            --tag-from-label rdo_version)
      echo $tag
      openstack overcloud container image prepare --tag $tag

    Change-Id: I12b16cb267c80e3059786fb980178eb5b3d1a76d
    Depends-On: I27ea031287604d70032fb5392aecbce313d4b096
    Closes-Bug: #1708967

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/506684

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/506684
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=f114a6194b67599a55707474bac3f98dda28791e
Submitter: Jenkins
Branch: stable/pike

commit f114a6194b67599a55707474bac3f98dda28791e
Author: Steve Baker <email address hidden>
Date: Mon Aug 28 11:20:06 2017 +1200

    Command to discover the versioned tag from latest

    This change implements the command
    "openstack overcloud container image tag discover" which will become
    part of the update and upgrade workflow for deploying new container
    images.

    It is used to discover the version-based tag by inspecting the image
    from a stable tag like current-tripleo-rdo. Stable tags like 'latest'
    or 'pike' can't be used for container updates because something needs
    to change to trigger the new containers being pulled. Without this
    command it would be up to the user to find out what versioned tag to
    specify when calling prepare.

    This was implemented as its own command instead of integrating with
    prepare because there may be multiple image build chains feeding into
    the image list (such as RDO, ceph and eventually others) and each will
    need its own call to discover the versioned tag.

      tag=$(openstack overcloud container image tag discover \
            --image trunk.registry.rdoproject.org/pike/centos-binary-base:current-tripleo-rdo
            --tag-from-label rdo_version)
      echo $tag
      openstack overcloud container image prepare --tag $tag

    Change-Id: I12b16cb267c80e3059786fb980178eb5b3d1a76d
    Depends-On: I27ea031287604d70032fb5392aecbce313d4b096
    Closes-Bug: #1708967
    (cherry picked from commit eca5393ddcced5187b9d360a0b5edb96ca2a534e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 7.3.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 8.0.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/519953

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519953
Committed: https://git.openstack.org/cgit/openstack/tripleo-docs/commit/?id=97f89092acf93feb83bdd7d065d4923d69b64a78
Submitter: Zuul
Branch: master

commit 97f89092acf93feb83bdd7d065d4923d69b64a78
Author: Steve Baker <email address hidden>
Date: Wed Nov 15 15:53:20 2017 +1300

    Document use of 'tag discover' command

    Now that the registry trunk.registry.rdoproject.org has the required
    combination of tags and labels, the documentation can be updated to
    use the command "openstack overcloud container image tag discover" to
    convert the current-tripleo-rdo tag to the versioned tag that comes
    from the DLRN reference.

    Change-Id: I29c8e542a2c446eebc96fd08a12b58e85a4b7b22
    Related-Bug: #1708967