openstack overcloud deploy --plan-environment-file override is broken

[John Fulton]

Deployment fails when using the --plan-environment-file option because of a path error.

If you deploy with:

openstack overcloud -v deploy \
         --templates \
         -p /usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-params.yaml \

Then the deployment will fail with the message below complaining it cannot find the file

  /usr/share/openstack-tripleo-heat-templates//usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-params.yaml

It looks like the default THT path has been appended even in cases where it shouldn't be.

Omiting the default THT path is not a option to workaround this because if you use -p without /usr/share/openstack-tripleo-heat-templates/ then the deployment fails because it cannot find the override.

This option is necessary to support derived paramters.

PLAYBOOK: cli-create-deployment-plan.yaml **************************************
1 plays in /usr/share/ansible/tripleo-playbooks/cli-create-deployment-plan.yaml

PLAY [Create Deployment Plan] **************************************************
2020-09-01 19:17:16.812980 | 2442011c-399d-791d-70aa-00000000000a | TASK | Set facts fro
m environment
2020-09-01 19:17:16.848304 | 2442011c-399d-791d-70aa-00000000000a | OK | Set facts fro
m environment | localhost | result={
    "censored": "the output has been hidden due to the fact that 'no_log: true' was specified
for this result",
    "changed": false
}
...
2020-09-01 19:17:16.996533 | 2442011c-399d-791d-70aa-000000000012 | TASK | Create plan $
nvironment if does not exit
2020-09-01 19:17:16.997035 | 2442011c-399d-791d-70aa-000000000010 | TIMING | Upload defau$
t templates | 0:00:00.238 | 0.03s
Using module file /usr/lib/python3.6/site-packages/ansible/modules/cloud/openstack/os_object.$
y
Pipelining is enabled.
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: stack
<127.0.0.1> EXEC /bin/sh -c '/usr/libexec/platform-python && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "<stdin>", line 102, in <module>
  File "<stdin>", line 94, in _ansiballz_main
  File "<stdin>", line 40, in invoke_module
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_os_object_payload_xq1xyewg/ansible_os_object_payload.zip/ansible/modules/
cloud/openstack/os_object.py", line 125, in <module>
  File "/tmp/ansible_os_object_payload_xq1xyewg/ansible_os_object_payload.zip/ansible/modules/
cloud/openstack/os_object.py", line 117, in main
  File "/tmp/ansible_os_object_payload_xq1xyewg/ansible_os_object_payload.zip/ansible/modules/
cloud/openstack/os_object.py", line 89, in process_object
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_object_store.py", line 297, in is_ob
ject_stale
    (file_md5, file_sha256) = self._get_file_hashes(filename)
  File "/usr/lib/python3.6/site-packages/openstack/cloud/_object_store.py", line 203, in _get_
file_hashes
    mtime=os.stat(filename).st_mtime)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-tem
plates//usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-param
s.yaml'
2020-09-01 19:17:18.423153 | 2442011c-399d-791d-70aa-000000000012 | FATAL | Create plan e
nvironment if does not exit | localhost | error={
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <mo
dule>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke
_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_mo
dule_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", l
ine 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib
64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ans
ible_os_object_payload_xq1xyewg/ansible_os_object_payload.zip/ansible/modules/cloud/openstack/
os_object.py\", line 125, in <module>\n File \"/tmp/ansible_os_object_payload_xq1xyewg/ansibl
e_os_object_payload.zip/ansible/modules/cloud/openstack/os_object.py\", line 117, in main\n F
ile \"/tmp/ansible_os_object_payload_xq1xyewg/ansible_os_object_payload.zip/ansible/modules/cl
oud/openstack/os_object.py\", line 89, in process_object\n File \"/usr/lib/python3.6/site-pac
kages/openstack/cloud/_object_store.py\", line 297, in is_object_stale\n (file_md5, file_sh
a256) = self._get_file_hashes(filename)\n File \"/usr/lib/python3.6/site-packages/openstack/c
loud/_object_store.py\", line 203, in _get_file_hashes\n mtime=os.stat(filename).st_mtime)\
nFileNotFoundError: [Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-te
mplates//usr/share/openstack-tripleo-heat-templates/plan-samples/plan-environment-derived-para
ms.yaml'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/749416

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/749416
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=295555839c5e1d8e7e5ab1bb3fb39537d09da989
Submitter: Zuul
Branch: master

commit 295555839c5e1d8e7e5ab1bb3fb39537d09da989
Author: John Fulton <email address hidden>
Date: Tue Sep 1 21:38:01 2020 -0400

    Only concatenate default_templates_dir when necessary

    By default the default_templates_dir is appended to the
    plan_environment file. This is fine unless the deployer
    overrides the default plan_environment file with the -p
    option. This patch adds jinja2 filters so that if the
    plan_environment starts with a '/', then the default
    templates directory is not appended. Otherwise the default
    behavior is unchanged.

    Change-Id: I5341da7fdea0a53b86295d74d68ef2013efb0ea0
    Closes-Bug: #1893861

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/749811

[John Fulton (jfulton-org)]

This fix can be backported to ussuri:

 https://github.com/openstack/tripleo-ansible/blob/stable/ussuri/tripleo_ansible/playbooks/cli-create-deployment-plan.yaml#L97

but not train as it didn't exist back then:

 https://github.com/openstack/tripleo-ansible/tree/stable/train/tripleo_ansible/playbooks

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/749811
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=a7b611f5a9c28c732d9385809af21abb21eace03
Submitter: Zuul
Branch: stable/ussuri

commit a7b611f5a9c28c732d9385809af21abb21eace03
Author: John Fulton <email address hidden>
Date: Tue Sep 1 21:38:01 2020 -0400

    Only concatenate default_templates_dir when necessary

    By default the default_templates_dir is appended to the
    plan_environment file. This is fine unless the deployer
    overrides the default plan_environment file with the -p
    option. This patch adds jinja2 filters so that if the
    plan_environment starts with a '/', then the default
    templates directory is not appended. Otherwise the default
    behavior is unchanged.

    Change-Id: I5341da7fdea0a53b86295d74d68ef2013efb0ea0
    Closes-Bug: #1893861
    (cherry picked from commit 295555839c5e1d8e7e5ab1bb3fb39537d09da989)