tripleo-upgrade infrared plugin , validate_oc_images_containers can't detect running openstack-ovn-northd container and errors out

[pini]

I'm seeing this issue in a composable roles jobs osp15 deployment:

Overcloud update goes ok , but post verification falsly errors out on :

2019-09-18 14:31:09 | PCMKLATEST container for image '192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1' is not running

(undercloud) [stack@undercloud-0 ~]$ ansible controller -b -mshell -a'docker ps|grep ovn-northd'
 [WARNING]: Found both group and host with same name: undercloud

controller-1 | CHANGED | rc=0 >>
0d0d434913a7 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1 dumb-init --singl... 6 hours ago Up 6 hours ago ovn-dbs-bundle-podman-1Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.

controller-2 | CHANGED | rc=0 >>
50081cbe1acd 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1 dumb-init --singl... 6 hours ago Up 6 hours ago ovn-dbs-bundle-podman-2Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.

controller-0 | CHANGED | rc=0 >>
4932c413e449 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1 dumb-init --singl... 6 hours ago Up 6 hours ago ovn-dbs-bundle-podman-0Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.

(undercloud) [stack@undercloud-0 ~]$ tail validate_oc_images_containers.log
2019-09-18 14:31:04 | Image 192.168.24.1:8787/rhosp15/openstack-iscsid:20190913.1 present in /home/stack/composable_roles/docker-images.yaml
2019-09-18 14:31:04 | Image 192.168.24.1:8787/rhosp15/openstack-horizon:20190913.1 present in /home/stack/composable_roles/docker-images.yaml
2019-09-18 14:31:04 | Image 192.168.24.1:8787/rhosp15/openstack-memcached:20190913.1 present in /home/stack/composable_roles/docker-images.yaml
2019-09-18 14:31:07 | ================================================================================
2019-09-18 14:31:07 | Validate PCMKLATEST docker images at host 192.168.24.32
2019-09-18 14:31:07 | ================================================================================
2019-09-18 14:31:08 | Image 192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1 present in /home/stack/composable_roles/docker-images.yaml
2019-09-18 14:31:09 | Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
2019-09-18 14:31:09 | PCMKLATEST container for image '192.168.24.1:8787/rhosp15/openstack-ovn-northd:20190913.1' is not running
2019-09-18 14:31:09 | Running container is ''
(undercloud) [stack@undercloud-0 ~]$ cat core_puddle_version
RHOS_TRUNK-15.0-RHEL-8-20190913.n.3(undercloud) [stack@undercloud-0 ~]$

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/687258

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/687299

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687258
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=72dd2c49e37db7ed014f297e3b7aad8dc22f60fe
Submitter: Zuul
Branch: master

commit 72dd2c49e37db7ed014f297e3b7aad8dc22f60fe
Author: Sergii Golovatiuk <email address hidden>
Date: Tue Oct 8 11:46:59 2019 +0200

    Disable overcloud image validation

    To speed up CI and development process it would be better to disable
    this script. CI will fail anyway even without this validation. Users
    will be able to enable it if/when they decides to use tripleo-upgrade
    for production upgrade.

    Closes-bug: #1844567
    Change-Id: Ia5a767491c2f297b396c5cc937c1495e4267a4e3

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/687299
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=70950225940c5a3c0ae66312abc63c2ef7bf4a27
Submitter: Zuul
Branch: stable/stein

commit 70950225940c5a3c0ae66312abc63c2ef7bf4a27
Author: Sergii Golovatiuk <email address hidden>
Date: Tue Oct 8 11:46:59 2019 +0200

    Disable overcloud image validation

    To speed up CI and development process it would be better to disable
    this script. CI will fail anyway even without this validation. Users
    will be able to enable it if/when they decides to use tripleo-upgrade
    for production upgrade.

    Closes-bug: #1844567
    Change-Id: Ia5a767491c2f297b396c5cc937c1495e4267a4e3

    (cherry picked from commit 72dd2c49e37db7ed014f297e3b7aad8dc22f60fe)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-upgrade 5.0.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/694133

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/694154

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694133
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=05eb80bb0d0ae10d74151b9c3977b94d81160b3b
Submitter: Zuul
Branch: stable/rocky

commit 05eb80bb0d0ae10d74151b9c3977b94d81160b3b
Author: Sergii Golovatiuk <email address hidden>
Date: Tue Oct 8 11:46:59 2019 +0200

    Disable overcloud image validation

    To speed up CI and development process it would be better to disable
    this script. CI will fail anyway even without this validation. Users
    will be able to enable it if/when they decides to use tripleo-upgrade
    for production upgrade.

    Closes-bug: #1844567
    Change-Id: Ia5a767491c2f297b396c5cc937c1495e4267a4e3
    (cherry picked from commit 72dd2c49e37db7ed014f297e3b7aad8dc22f60fe)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/694154
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=7d769026cefcd6edb782383d6a7d641e0e70f535
Submitter: Zuul
Branch: stable/queens

commit 7d769026cefcd6edb782383d6a7d641e0e70f535
Author: Sergii Golovatiuk <email address hidden>
Date: Tue Oct 8 11:46:59 2019 +0200

    Disable overcloud image validation

    To speed up CI and development process it would be better to disable
    this script. CI will fail anyway even without this validation. Users
    will be able to enable it if/when they decides to use tripleo-upgrade
    for production upgrade.

    Closes-bug: #1844567
    Change-Id: Ia5a767491c2f297b396c5cc937c1495e4267a4e3
    (cherry picked from commit 72dd2c49e37db7ed014f297e3b7aad8dc22f60fe)
    (cherry picked from commit 87639f0badbb5873f03d5c69ae75b0c828464f55)