AnsibleParserError with containerised undercloud install

[Julie Pichon]

Since Friday, whether I try to install the undercloud on a brand new VM or upgrade my existing one, it fails with the following error after the stack creation completes:

$ openstack undercloud install --use-heat
[...]
2018-04-30 08:20:37Z [undercloud]: CREATE_COMPLETE Stack CREATE completed successfully
** Downloading undercloud ansible.. **
Not cleaning temporary directory /home/stack/tripleoclient-TcP1wm
** Running ansible.. **
 [WARNING]: Skipping unexpected key (hostvars) in group (_meta), only "vars", "children" and "hosts" are
valid
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static
 inclusions or 'include_tasks' for dynamic inclusions. This feature will be removed in a future release.
 Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module
documentation details page may explain more about this rationale.. This feature will be removed in a
future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
exception type: <class 'ansible.errors.AnsibleParserError'>
exception: no action detected in task. This often indicates a misspelled module name, or incorrect module path.
Not cleaning temporary directory /home/stack/tripleoclient-TcP1wm

http://paste.openstack.org/show/720113/

Running with verbose ($ openstack undercloud upgrade --use-heat --verbose) I can see the ansible task it's failing on:

** Downloaded undercloud ansible to /home/stack/tripleo-tRYKUU-config **
Not cleaning temporary directory /home/stack/tripleoclient-uBSrcD
** Running ansible.. **
Running Ansible: ansible-playbook -i /home/stack/tripleo-tRYKUU-config/inventory.yaml deploy_steps_playbook.yaml -e role_name=Undercloud -e deploy_server_id=undercloud -e bootstrap_server_id=undercloud

http://paste.openstack.org/show/720122/

It doesn't work when I run the command directly either, though the error looks different - seems not to be picking up the host correctly but I'm not sure if that's actually related to what I'm seeing on undercloud install/upgrade or just due to running the command differently.

[WARNING]: Could not match supplied host pattern, ignoring: undercloud

PLAY [Gather facts undercloud] ************************************************************************************************************************************************************************************
skipping: no hosts matched
[...]

http://paste.openstack.org/show/720121/

Nothing jumps out in the heat logs or undercloud install logs. I'm following the steps at https://docs.openstack.org/tripleo-docs/latest/install/installation/installation.html to install/update.

python-tripleoclient-10.0.1-0.20180428084938.3ad2abb.el7.centos.noarch
python-tripleoclient-heat-installer-10.0.1-0.20180428084938.3ad2abb.el7.centos.noarch
openstack-tripleo-heat-templates-9.0.0-0.20180428235950.a3a1ed5.el7.centos.noarch

[Bogdan Dobrelya (bogdando)]

The issue seems related to the recent changes for using hostnames in tripleo inventory?

[Bogdan Dobrelya (bogdando)]

Do you perform an upgrade from instack? Could you please also attach those produced tripleo-* directory and the heat env files used with the deploy/upgrade commands?

[Julie Pichon (jpichon)]

I only ever installed containerised undercloud on these two VMs (fresh one this morning), there was no upgrade from instack. I'm using the $ openstack undercloud install --use-heat / $ openstack undercloud upgrade --use-heat commands, but I can see on the failure that that's it's calling deploy with the following arguments:

Command '['sudo', 'openstack', 'undercloud', 'deploy', '--local-domain=localdomain', '--local-ip=192.168.24.1/24', '--templates=/usr/share/openstack-tripleo-heat-templates/', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/lifecycle/undercloud-upgrade-prepare.yaml', '--heat-native', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/mistral.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/zaqar.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tripleo-ui.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/tempest.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/public-tls-undercloud.yaml', '--public-virtual-ip', '192.168.24.2', '--control-virtual-ip', '192.168.24.3', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/use-dns-for-vips.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-haproxy.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/services/undercloud-keepalived.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/docker.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/config-download-environment.yaml', '-e', '/usr/share/openstack-tripleo-heat-templates/environments/undercloud.yaml', '-e', '/tmp/undercloud_parameters.yaml', '--output-dir=/home/stack', '--log-file=/home/stack/install-undercloud.log']' returned non-zero exit status 4

[Emilien Macchi (emilienm)]

Let me re-open it, I could reproduce Julie's bug in my environment but haven't found the root cause yet. The problem I had was an empty task in Undercloud/host_prep_tasks.yml at the very end (empty hash) which caused Ansible to fail.

Let's keep the bug open until we confirm we confirm it works now.

[Alex Schultz (alex-schultz)]

I hit this on a fresh install as well.
[centos@undercloud ~]$ rpm -qa | grep tripleo-
openstack-tripleo-heat-templates-9.0.0-0.20180427131225.f4fa117.el7.centos.noarch
openstack-tripleo-common-containers-9.0.2-0.20180427191251.79af8c0.el7.centos.noarch
openstack-tripleo-common-9.0.2-0.20180427191251.79af8c0.el7.centos.noarch
ansible-tripleo-ipsec-8.1.1-0.20180405121919.325d233.el7.centos.noarch
openstack-tripleo-puppet-elements-9.0.0-0.20180427082845.fae8944.el7.centos.noarch
puppet-tripleo-9.0.1-0.20180426102317.6b9f8a5.el7.centos.noarch
python2-tripleo-repos-0.0.1-0.20180418175107.ef4e12e.el7.centos.noarch
openstack-tripleo-image-elements-9.0.0-0.20180426101324.08096ad.el7.centos.noarch

I will retry with the latest code from tripleo to see if it's still a problem

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/565297

[Julie Pichon (jpichon)]

https://review.openstack.org/#/c/565291/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565291
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=ca69e1c57dcb60874632c33e0aa51fb0b83775e8
Submitter: Zuul
Branch: master

commit ca69e1c57dcb60874632c33e0aa51fb0b83775e8
Author: Emilien Macchi <email address hidden>
Date: Mon Apr 30 12:54:53 2018 -0700

    Remove empty host prep tasks for Tempest

    There is no task, so we don't need this config, which leads to Ansible
    error if a task is empty.

    Change-Id: I8ca0d6f012e3bac68d052df117366d816c2b9cb6
    Closes-Bug: #1768019

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565297
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=960fc05b8a6f1ebd4d5fad6c216eb0fc2ea46f8f
Submitter: Zuul
Branch: master

commit 960fc05b8a6f1ebd4d5fad6c216eb0fc2ea46f8f
Author: Alex Schultz <email address hidden>
Date: Mon Apr 30 14:22:48 2018 -0600

    Ensure tasks aren't empty dicts

    Ansible doesn't like it when tasks are empty dicts so let's make sure
    we aren't passing an empty dict in to the ansible tasks. Unfortunately
    since we're validating non-heat processed yaml, the list_concats end
    up being dicts to the yaml validator so we can't just ensure they are
    always lists.

    Change-Id: I13dcc4f3d4ea2ac4c7ab62887dfea293246f6530
    Related-Bug: #1768019

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/565645

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565645
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=081a111f90eb62b47801dd00ecff6214649862aa
Submitter: Zuul
Branch: stable/queens

commit 081a111f90eb62b47801dd00ecff6214649862aa
Author: Alex Schultz <email address hidden>
Date: Mon Apr 30 14:22:48 2018 -0600

    Ensure tasks aren't empty dicts

    Ansible doesn't like it when tasks are empty dicts so let's make sure
    we aren't passing an empty dict in to the ansible tasks. Unfortunately
    since we're validating non-heat processed yaml, the list_concats end
    up being dicts to the yaml validator so we can't just ensure they are
    always lists.

    Change-Id: I13dcc4f3d4ea2ac4c7ab62887dfea293246f6530
    Related-Bug: #1768019
    (cherry picked from commit 960fc05b8a6f1ebd4d5fad6c216eb0fc2ea46f8f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b3 development milestone.