tripleo container image prepare fails with -e to a relative location

[Alex Schultz]

Regression from the conversion to use ansible.

[centos@undercloud ~]$ sudo openstack tripleo container image prepare -e container-image-prepare.yaml |& tee cip.log

PLAY [Container Image Prepare] *************************************************
2020-09-10 19:00:44.933797 | fa163eba-f55b-5bf5-43b9-000000000008 | TASK | Fetch roles_data
2020-09-10 19:00:45.250202 | fa163eba-f55b-5bf5-43b9-000000000008 | OK | Fetch roles_data | localhost
2020-09-10 19:00:45.255011 | fa163eba-f55b-5bf5-43b9-000000000009 | TASK | Set fact for log file
2020-09-10 19:00:45.255591 | fa163eba-f55b-5bf5-43b9-000000000008 | TIMING | Fetch roles_data | 0:00:00.374 | 0.32s
2020-09-10 19:00:45.280556 | fa163eba-f55b-5bf5-43b9-000000000009 | OK | Set fact for log file | localhost
2020-09-10 19:00:45.284324 | fa163eba-f55b-5bf5-43b9-00000000000a | TASK | Get all files in directories
2020-09-10 19:00:45.284831 | fa163eba-f55b-5bf5-43b9-000000000009 | TIMING | Set fact for log file | 0:00:00.403 | 0.03s
[DEPRECATION WARNING]: evaluating 'environment_directories' as a bare variable,
 this behaviour will go away and you might need to add |bool to the expression
in the future. Also see CONDITIONAL_BARE_VARS configuration toggle. This
feature will be removed in version 2.12. Deprecation warnings can be disabled
by setting deprecation_warnings=False in ansible.cfg.
2020-09-10 19:00:45.596004 | fa163eba-f55b-5bf5-43b9-00000000000a | OK | Get all files in directories | localhost
2020-09-10 19:00:45.600519 | fa163eba-f55b-5bf5-43b9-00000000000b | TASK | Build environment_files
2020-09-10 19:00:45.601080 | fa163eba-f55b-5bf5-43b9-00000000000a | TIMING | Get all files in directories | 0:00:00.720 | 0.32s
2020-09-10 19:00:45.639243 | fa163eba-f55b-5bf5-43b9-00000000000b | OK | Build environment_files | localhost
2020-09-10 19:00:45.643139 | fa163eba-f55b-5bf5-43b9-00000000000c | TASK | Build heat stack environment
2020-09-10 19:00:45.643683 | fa163eba-f55b-5bf5-43b9-00000000000b | TIMING | Build environment_files | 0:00:00.762 | 0.04s
2020-09-10 19:00:46.913514 | fa163eba-f55b-5bf5-43b9-00000000000c | FATAL | Build heat stack environment | localhost | error={"changed": false, "environment": {}, "error": "<urlopen error [Errno 2] No such file or directory: '/usr/share/ansible/tripleo-playbooks/container-image-prepare.yaml'>", "msg": "Error buiding environment: <urlopen error [Errno 2] No such file or directory: '/usr/share/ansible/tripleo-playbooks/container-image-prepare.yaml'>", "success": false}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
localhost : ok=4 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
2020-09-10 19:00:46.917641 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-09-10 19:00:46.918020 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:00:02.036 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-09-10 19:00:46.918382 | UUID | Info | Task Name | Run Time
2020-09-10 19:00:46.918723 | fa163eba-f55b-5bf5-43b9-00000000000c | SUMMARY | Build heat stack environment | 1.27s
2020-09-10 19:00:46.919097 | fa163eba-f55b-5bf5-43b9-000000000008 | SUMMARY | Fetch roles_data | 0.32s
2020-09-10 19:00:46.919453 | fa163eba-f55b-5bf5-43b9-00000000000a | SUMMARY | Get all files in directories | 0.32s
2020-09-10 19:00:46.919767 | fa163eba-f55b-5bf5-43b9-00000000000b | SUMMARY | Build environment_files | 0.04s
2020-09-10 19:00:46.920117 | fa163eba-f55b-5bf5-43b9-000000000009 | SUMMARY | Set fact for log file | 0.03s
2020-09-10 19:00:46.920497 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-09-10 19:00:46.920853 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-09-10 19:00:46.921228 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~
2020-09-10 19:00:46.921660 | The following node(s) had failures: localhost
2020-09-10 19:00:46.922030 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-container-image-prepare.yaml, Run Status: failed, Return Code: 2
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py", line 1078, in take_action
    extra_vars=extra_vars)
  File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 699, in run_ansible_playbook
    raise RuntimeError(err_msg)
RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-container-image-prepare.yaml, Run Status: failed, Return Code: 2
Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-container-image-prepare.yaml, Run Status: failed, Return Code: 2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/751087

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/751100

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: master
Review: https://review.opendev.org/751087
Reason: https://review.opendev.org/#/c/751100/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/751165

[Alex Schultz (alex-schultz)]

Backport should not be necessary as it's a side effect of the conversion to ansible

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/751165
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=afe6f82185be50ea4d839f648d2496591c08c026
Submitter: Zuul
Branch: master

commit afe6f82185be50ea4d839f648d2496591c08c026
Author: Rabi Mishra <email address hidden>
Date: Fri Sep 11 08:58:55 2020 +0530

    Add filter for absolute path of file/dir

    This adds a custom filter that can be used in playbooks
    to convert relative file/dir paths wrt home dir and
    tht_root to absolute paths.

    Also changes relevant playbooks to use it.

    Closes-Bug: #1895194
    Change-Id: I260fa3ec2f85a095d1d3ae4c22e2488d6000257a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/751100
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=5abe17dc4a7d0d36ca792a0c8d7ceece459789b3
Submitter: Zuul
Branch: master

commit 5abe17dc4a7d0d36ca792a0c8d7ceece459789b3
Author: Alex Schultz <email address hidden>
Date: Thu Sep 10 14:19:19 2020 -0600

    Fix pathing for container image prepare

    When we pass the paths to the ansible playbook, they need to be absolute
    paths as the relative cwd changes when it runs.

    Change-Id: I8a434e38eb1a69530190df96d3b555ea14e4488a
    Closes-Bug: #1895194

[Rafael Folco (rafaelfolco)]

@marios,

I think we are hitting this bug when we install tripleo repos (python-tripleoclient, tripleo-ansible, and tripleo-common) from pip in a virtualenv.

 No such playbook: /usr/share/ansible/tripleo-playbooks/cli-container-image-prepare.yaml

The file in a venv is:
{{ venv }}/share/ansible/tripleo-playbooks/cli-container-image-prepare.yaml

The broken job is mol-tripleo_common_integration - https://review.rdoproject.org/r/#/c/28081/

image prepare is called from inside venv here:
https://github.com/rdo-infra/ci-config/blob/master/molecule/tripleo_common_integration/converge.yml

[Marios Andreou (marios-b)]

@folco ack thanks for pointing me to this bug and comment #8 above.

agree this is definitely the same issue we are facing with mol-tripleo_common_integration - https://review.rdoproject.org/r/#/c/28081/

the job started failing after merge of https://review.opendev.org/#/c/746477/ & Depends-On: https://review.opendev.org/#/c/746476/

---

trying to understand though why we are still hitting it with pip install from git source at https://review.rdoproject.org/r/#/c/28081/83..84/molecule/tripleo_common_integration/prepare.yml so we *should* be running with the fixes @ * https://review.opendev.org/#/c/751100/ * https://review.opendev.org/#/c/751165/5

[Rafael Folco (rafaelfolco)]

I took a quick look at this issue and this doesn't sound right to me:
https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/container_image.py#L1094

if we found relative path file, why we are using absolute path there ?

[Alex Schultz (alex-schultz)]

This issue was for the values being sent to the playbook execution. Your issue is the cli playbook lookup is not dynamic (or relative) for execution. The jobs were relying on tripleoclient having all the necessary items to invoke this command. That is no longer true since the cli playbook now comes from tripleo-ansible which the location may not match what is distributed via rpms. This command likely needs some logic like https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v2/tripleo_container_image.py#L39-L42 for the playbook locations to support their installation into a venv