tripleoclient doesn't check if network file exists prior creating/uploading plan

[Cédric Jeanneret]

Hello,

when we pass "-n non-existing-file.yaml" to `openstack overcloud deploy' command, it doesn't check if the file exists or not. This leads to a broken swift container content, and spits ugly error:

Output env file exists, moving it to backup.
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action
    self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir
    new_tht_root, tht_root)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates
    validate_stack=False)
  File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 232, in update_plan_from_templates
    plan_env_file, networks_file)
  File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 55, in _upload_templates
    constants.OVERCLOUD_NETWORKS_FILE, networks_file)
  File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 280, in _upload_file
    with open(local_filename, 'rb') as file_content:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml'
[Errno 2] No such file or directory: '/usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry-11.yaml'
Removing the current plan files
Uploading new plan files

and when we correct and want to re-run:
Output env file exists, moving it to backup.
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 32, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 925, in take_action
    self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 374, in _deploy_tripleo_heat_templates_tmpdir
    new_tht_root, tht_root)
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 400, in _deploy_tripleo_heat_templates
    validate_stack=False)
  File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 206, in update_plan_from_templates
    passwords = _load_passwords(swift_client, name)
  File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/plan_management.py", line 293, in _load_passwords
    return plan_env['passwords']
KeyError: 'passwords'
'passwords'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/667562

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/667562
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=bd354f1fc8ae7a36898a22e01e52815d85d8378d
Submitter: Zuul
Branch: master

commit bd354f1fc8ae7a36898a22e01e52815d85d8378d
Author: Cédric Jeanneret <email address hidden>
Date: Wed Jun 26 11:24:14 2019 +0200

    Ensure files exist before building/uploading plan

    Until now, only environment files were tested for their eXistenZ. This
    patch corrects this issue, allowing to get a graceful failure before
    plan is compiled and uploaded.

    The following files are now checked, and deploy will gracefully fail in
    case of non-existing file:
    - plan-environment-file
    - answers-file
    - networks-file

    No Bioport nor Pod was harmed for this patch.

    Change-Id: Ifd4229b68571fd6b91574675c1f04d7b5c755eab
    Closes-Bug: #1834307

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/668596

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668596
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=f78809a3e3122f268bcc8fa1a9c12974735654b9
Submitter: Zuul
Branch: stable/stein

commit f78809a3e3122f268bcc8fa1a9c12974735654b9
Author: Cédric Jeanneret <email address hidden>
Date: Wed Jun 26 11:24:14 2019 +0200

    Ensure files exist before building/uploading plan

    Until now, only environment files were tested for their eXistenZ. This
    patch corrects this issue, allowing to get a graceful failure before
    plan is compiled and uploaded.

    The following files are now checked, and deploy will gracefully fail in
    case of non-existing file:
    - plan-environment-file
    - answers-file
    - networks-file

    No Bioport nor Pod was harmed for this patch.

    Change-Id: Ifd4229b68571fd6b91574675c1f04d7b5c755eab
    Closes-Bug: #1834307
    (cherry picked from commit bd354f1fc8ae7a36898a22e01e52815d85d8378d)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/668689

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668689
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=cb976f672959f3a14360390d6bc8b6ba8e873fa2
Submitter: Zuul
Branch: stable/rocky

commit cb976f672959f3a14360390d6bc8b6ba8e873fa2
Author: Cédric Jeanneret <email address hidden>
Date: Wed Jun 26 11:24:14 2019 +0200

    Ensure files exist before building/uploading plan

    Until now, only environment files were tested for their eXistenZ. This
    patch corrects this issue, allowing to get a graceful failure before
    plan is compiled and uploaded.

    The following files are now checked, and deploy will gracefully fail in
    case of non-existing file:
    - plan-environment-file
    - answers-file
    - networks-file

    No Bioport nor Pod was harmed for this patch.

    Change-Id: Ifd4229b68571fd6b91574675c1f04d7b5c755eab
    Closes-Bug: #1834307
    (cherry picked from commit bd354f1fc8ae7a36898a22e01e52815d85d8378d)
    (cherry picked from commit f78809a3e3122f268bcc8fa1a9c12974735654b9)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/668940

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/668940
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=9e5ec97754eeb152fb92e5a27904bb8eb1378db4
Submitter: Zuul
Branch: stable/queens

commit 9e5ec97754eeb152fb92e5a27904bb8eb1378db4
Author: Cédric Jeanneret <email address hidden>
Date: Wed Jun 26 11:24:14 2019 +0200

    Ensure files exist before building/uploading plan

    Until now, only environment files were tested for their eXistenZ. This
    patch corrects this issue, allowing to get a graceful failure before
    plan is compiled and uploaded.

    The following files are now checked, and deploy will gracefully fail in
    case of non-existing file:
    - plan-environment-file
    - answers-file
    - networks-file

    No Bioport nor Pod was harmed for this patch.

    Change-Id: Ifd4229b68571fd6b91574675c1f04d7b5c755eab
    Closes-Bug: #1834307
    (cherry picked from commit bd354f1fc8ae7a36898a22e01e52815d85d8378d)
    (cherry picked from commit f78809a3e3122f268bcc8fa1a9c12974735654b9)
    (cherry picked from commit cb976f672959f3a14360390d6bc8b6ba8e873fa2)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 11.5.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 12.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 10.7.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 9.3.1 release.