standalone-upgrade job fails in post-upgrade tasks for " Failed to import docker or docker-py - No module named docker"

[Marios Andreou]

In [1] we disable the validations on tripleo-ci-centos-7-standalone-upgrade job to temporarily work around https://launchpad.net/bugs/1823882 . However you can see at [2] that the post-upgrade steps fail for during post-upgrade tasks with trace like

 2019-04-09 13:48:57 | 2019-04-09 13:48:57.934 92611 WARNING tripleoclient.v1.tripleo_upgrade.Upgrade [-] failed: [standalone] (item=cinder_api) => {"changed": false, "item": "cinder_api", "msg": "Failed to import docker or docker-py - No module named docker. Try `pip install docker` or `pip install docker-py` (Python 2.6)"}
 2019-04-09 13:48:58 | 2019-04-09 13:48:58.251 92611 WARNING tripleoclient.v1.tripleo_upgrade.Upgrade [-] failed: [standalone] (item=cinder_api_cron) => {"changed": false, "item": "cinder_api_cron", "msg": "Failed to import docker or docker-py - No module named docker. Try `pip install docker` or `pip install docker-py` (Python 2.6)"}

 2019-04-09 13:48:58 | 2019-04-09 13:48:58.365 92611 ERROR tripleoclient.v1.tripleo_upgrade.Upgrade [-] Exception: Post Upgrade failed: DeploymentError: Post Upgrade failed
 2019-04-09 13:48:58 | Traceback (most recent call last):
 2019-04-09 13:48:58 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1262, in _standalone_deploy
 2019-04-09 13:48:58 | raise exceptions.DeploymentError('Post Upgrade failed')
 2019-04-09 13:48:58 | DeploymentError: Post Upgrade failed

Looks like the post-upgrade tasks from cinder [3] are failing and the trace suggests a problem with tripleo-docker-rm [4] - see the requirements in

[1] https://review.openstack.org/#/c/651213/
[2] http://logs.openstack.org/13/651213/1/check/tripleo-ci-centos-7-standalone-upgrade/d3862ed/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz
[3] https://github.com/openstack/tripleo-heat-templates/blob/a062f1d27dcaf4e00f5940b2caa68081a7453bf4/deployment/cinder/cinder-api-container-puppet.yaml#L381-L388
[4] https://github.com/openstack/tripleo-common/blob/master/roles/tripleo-docker-rm/tasks/main.yaml
[5] https://docs.ansible.com/ansible/latest/modules/docker_container_module.html#requirements

[Marios Andreou (marios-b)]

fix posted in https://review.openstack.org/#/c/651510/ pip install the docker-py seems to do the trick

job green in test at http://logs.openstack.org/13/651213/2/check/tripleo-ci-centos-7-standalone-upgrade/6242fc1/ (from https://review.openstack.org/#/c/651213/ but about to discuss that with Tengu we may abandon that one).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/651789

[OpenStack Infra (hudson-openstack)]

Change abandoned by Marios Andreou (<email address hidden>) on branch: master
Review: https://review.openstack.org/651510
Reason: going with https://review.openstack.org/#/c/651789/1

thanks for reviewing

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/651789
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=e368e15226b18d6f75b0ffaca424ce4cb574b2a4
Submitter: Zuul
Branch: master

commit e368e15226b18d6f75b0ffaca424ce4cb574b2a4
Author: Emilien Macchi <email address hidden>
Date: Thu Apr 11 09:57:00 2019 -0400

    docker-rm: check if rpm dependency is actually installed

    If the deployment wasn't deployed under Docker, python2-docker package
    is missing. Therefore, during the upgrade we don't want to try to remove
    the containers under Docker, since they don't exist anyway.

    python2-docker is needed by Ansible to remove the containers, so let's
    make sure the package is installed, which is the case when the
    deployment was under Docker.
    If the rpm doesn't exist, we'll skip the tasks and don't try to remove
    the containers.

    The main case where docker-rm is useful is when upgrading on centos7
    from rocky to stein, since we clear containers running in Docker during
    the upgrade, so they run on Podman.

    In reality, the future platform won't have Docker installed, so this
    role will become useless and we'll probably deprecate it.

    Closes-Bug: #1824301
    Change-Id: Ic5036ffa756775e3806e18b09e034af2290bfb56

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/655511

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/655511
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=f19863f332ca1e42c1a20462212c0a809d59241d
Submitter: Zuul
Branch: stable/stein

commit f19863f332ca1e42c1a20462212c0a809d59241d
Author: Emilien Macchi <email address hidden>
Date: Thu Apr 11 09:57:00 2019 -0400

    docker-rm: check if rpm dependency is actually installed

    If the deployment wasn't deployed under Docker, python2-docker package
    is missing. Therefore, during the upgrade we don't want to try to remove
    the containers under Docker, since they don't exist anyway.

    python2-docker is needed by Ansible to remove the containers, so let's
    make sure the package is installed, which is the case when the
    deployment was under Docker.
    If the rpm doesn't exist, we'll skip the tasks and don't try to remove
    the containers.

    The main case where docker-rm is useful is when upgrading on centos7
    from rocky to stein, since we clear containers running in Docker during
    the upgrade, so they run on Podman.

    In reality, the future platform won't have Docker installed, so this
    role will become useless and we'll probably deprecate it.

    Closes-Bug: #1824301
    Change-Id: Ic5036ffa756775e3806e18b09e034af2290bfb56
    (cherry picked from commit e368e15226b18d6f75b0ffaca424ce4cb574b2a4)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 11.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 10.8.0 release.