HA: new container naming scheme "cluster-common-tag" make deployment fail

[Damien Ciabrini]

Since [1,2], HA containers are now configured to use a new image name scheme that acts as an intermediate tag which enables changing container image name during a minor update without service disruption.

The feature has passed CI but was only exercised for docker, since currently we don't support podman in pacemaker on CentOS 7.0.

Unfortunately, podman has a slightly different semantics than docker when it comes to tagging images with arbitrary tag name, i.e. when doing the following high-level command:

# podman tag undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-rabbitmq:20191213.1 cluster-common-tag/rhosp16-openstack-rabbitmq-volume:pcmklatest

podman prepends 'localhost/' in front of the new tag:

# podman images | grep cluster
localhost/cluster-common-tag/rhosp16-openstack-rabbitmq-volume pcmklatest 10bb0d557540 3 weeks ago 596 MB

This in itself is not a big deal because one can still run podman commands without passing "localhost/" for the image name:

# podman run --rm --net=host -it cluster-common-tag/rhosp16-openstack-rabbitmq-volume:pcmklatest /bin/ls /
bin boot dev etc home lib lib64 lost+found media mnt openstack opt proc root run sbin srv sys tmp usr var

The problem is that in pacemaker, the podman resource agent uses regular expressions to check whether image tag 'cluster-common-tag/rhosp16-openstack-rabbitmq-volume:pcmklatest' exists in the container storage, and it cannot find it. So it refuses to start containers and the entire stack deployment fails.

[1] Id369154d147cd5cf0a6f997bf806084fc7580e01
[2] I7a63e8e2d9457c5025f3d70aeed6922e24958049

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/701388

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/701388
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b61f71a74a3953b25bc9849a8e7b82bd803dc03e
Submitter: Zuul
Branch: master

commit b61f71a74a3953b25bc9849a8e7b82bd803dc03e
Author: Damien Ciabrini <email address hidden>
Date: Tue Jan 7 16:22:49 2020 +0100

    HA: Fix the cluster common tag behaviour with podman

    Since [1,2], HA container image name used in pacemaker are
    look like container-common-tag/<servicename>:pcmklatest.

    Unlike docker, podman prepends a host information in
    front of this tag, which confuses the podman resource
    agent in pacemaker.

    Make the first part of the tag look like a fqdn so that
    podman doesn't change it.

    Closes-Bug: #1858648

    [1] Id369154d147cd5cf0a6f997bf806084fc7580e01
    [2] I7a63e8e2d9457c5025f3d70aeed6922e24958049

    Change-Id: I07d72b87225dbadbc4df46564452ccb232593219

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/701477

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/701477
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=8f93d271a009c568d2034d02d6216af3626b4425
Submitter: Zuul
Branch: stable/train

commit 8f93d271a009c568d2034d02d6216af3626b4425
Author: Damien Ciabrini <email address hidden>
Date: Tue Jan 7 16:22:49 2020 +0100

    HA: Fix the cluster common tag behaviour with podman

    Since [1,2], HA container image name used in pacemaker are
    look like container-common-tag/<servicename>:pcmklatest.

    Unlike docker, podman prepends a host information in
    front of this tag, which confuses the podman resource
    agent in pacemaker.

    Make the first part of the tag look like a fqdn so that
    podman doesn't change it.

    Closes-Bug: #1858648

    [1] Id369154d147cd5cf0a6f997bf806084fc7580e01
    [2] I7a63e8e2d9457c5025f3d70aeed6922e24958049

    Change-Id: I07d72b87225dbadbc4df46564452ccb232593219
    (cherry picked from commit b61f71a74a3953b25bc9849a8e7b82bd803dc03e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 12.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.