ocata->pike upgrade CI - ansible fails with "no action detected in task"

[Jiří Stránský]

This is an error we hit in upgrades CI but not when executing upgrades locally:

http://logs.openstack.org/84/450784/12/experimental/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/b1ac19e/logs/undercloud/home/jenkins/overcloud_upgrade_console.log.txt.gz#_2017-04-11_16_20_33

2017-04-11 16:20:33 | 2017-04-11 16:20:01Z [overcloud]: UPDATE_FAILED resources.AllNodesDeploySteps: Error: resources.ControllerUpgrade_Step0.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 4
2017-04-11 16:20:33 |
2017-04-11 16:20:33 | Stack overcloud UPDATE_FAILED
2017-04-11 16:20:33 |
2017-04-11 16:20:33 | overcloud.AllNodesDeploySteps.ControllerUpgrade_Step0.0:
2017-04-11 16:20:33 | resource_type: OS::Heat::SoftwareDeployment
2017-04-11 16:20:33 | physical_resource_id: 7878167f-dc7b-4231-adb6-dbd1631aa5c1
2017-04-11 16:20:33 | status: CREATE_FAILED
2017-04-11 16:20:33 | status_reason: |
2017-04-11 16:20:33 | Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
2017-04-11 16:20:33 | deploy_stdout: |
2017-04-11 16:20:33 |
2017-04-11 16:20:33 | deploy_stderr: |
2017-04-11 16:20:33 | ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
2017-04-11 16:20:33 | ++ onerror
2017-04-11 16:20:33 | ++ echo 'Command failed'
2017-04-11 16:20:33 | Command failed
2017-04-11 16:20:33 | ++ openstack stack resource list overcloud
2017-04-11 16:20:37 | ++ openstack stack failures list --long overcloud
2017-04-11 16:21:07 | ++ exit 1

[Jiří Stránský (jistr)]

Based on discussion with Artem Hrechanychenko and further investigation, this problem is likely caused by the fact that all multinode jobs deploy pacemaker environments.

The problem is probably twofold:

1) We're likely missing Pacemaker Ansible module on the overcloud, hence Ansible not recognizing the tasks correctly.

2) Currently we cannot upgrade Pacemakerized deployments anyway (containerization of pacemakerized services is WIP), upgrades have only been tested with non-Pacemaker deployments so far.

I'm going to focus the fix on problem 2 for now, to unblock the upgrade CI as soon as possible.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/456190

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/456190
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=4600843cffd1df92dde04923e14a9556d725c59b
Submitter: Jenkins
Branch: master

commit 4600843cffd1df92dde04923e14a9556d725c59b
Author: Jiri Stransky <email address hidden>
Date: Wed Apr 12 13:55:59 2017 +0200

    Non-ha multinode environment for container upgrade job

    Non-working containers upgrade CI is caused by the fact that all
    multinode jobs deploy pacemaker environments.

    Currently we cannot upgrade Pacemakerized deployments
    anyway (containerization of pacemakerized services is WIP), upgrades
    have only been tested with non-Pacemaker deployments so far.

    We need a new environment which will not try deploying in a
    pacemakerized way. When pacemaker-managed services are containerized, we
    can change the job to upgrade an HA deployment (or single-node "HA" at
    least), and perhaps even get rid of the environment file introduced
    here, and reuse multinode.yaml.

    Change-Id: Ie635b1b3a0b91ed5305f38d3c76f6a961efc1d30
    Closes-Bug: #1682051

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.0.0b2 development milestone.