openstack overcloud export expecting different directory structure for --config-download-dir

Bug #1884246 reported by John Fulton
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Rabi Mishra

Bug Description

`openstack overcloud export` fails when passed a valid argument for --config-download-dir with an error that it cannot find the group_vars/overcloud.json inside of config-download/STACK_NAME [1].

However, group_vars/overcloud.json is inside of config-download without the STACK_NAME subdirectory [2].

`openstack overcloud export` should not be appending STACK_NAME into the path.

The config-download directory was generated by running the following:

    openstack overcloud config download \
              --name control-plane \
              --config-dir config-download

Perhaps `openstack overcloud config download` was updated to use a different structure but `openstack overcloud export` was not.

This breaks DCN deployments.

[1]

(undercloud) [CentOS-8.1 - stack@undercloud dcn]$ openstack overcloud export --config-download-dir control-plane/config-download/ --stack control-plane --output-file /home/stack/control-plane-export.yaml
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_export.py", line 94, in take_action
    clients.orchestration, stack, False, config_download_dir))
  File "/usr/lib/python3.6/site-packages/tripleoclient/export.py", line 95, in export_stack
    with open(file, 'r') as ff:
FileNotFoundError: [Errno 2] No such file or directory: 'control-plane/config-download/control-plane/group_vars/overcloud.json'
[Errno 2] No such file or directory: 'control-plane/config-download/control-plane/group_vars/overcloud.json'
(undercloud) [CentOS-8.1 - stack@undercloud dcn]$

[2]

(undercloud) [CentOS-8.1 - stack@undercloud dcn]$ ls control-plane/config-download/control-plane/group_vars/overcloud.json
ls: cannot access 'control-plane/config-download/control-plane/group_vars/overcloud.json': No such file or directory
(undercloud) [CentOS-8.1 - stack@undercloud dcn]$

(undercloud) [CentOS-8.1 - stack@undercloud dcn]$ ls control-plane/config-download/group_vars/overcloud.json
control-plane/config-download/group_vars/overcloud.json
(undercloud) [CentOS-8.1 - stack@undercloud dcn]$

Revision history for this message
John Fulton (jfulton-org) wrote :

WORKAROUND

# Quick ugly hack:

mkdir $STACK/config-download/$STACK
pushd $STACK/config-download/$STACK/
ln -s ../group_vars group_vars
popd

# then run

openstack overcloud export \
          --config-download-dir $STACK/config-download/ \
          --stack $STACK \
          --output-file ~/${STACK}-export.yaml

Revision history for this message
Rabi Mishra (rabi) wrote :

It does not look related to mistral removal.

Looking at stable/train 'openstack overcloud config download' does not create a subfolder with the stack name in the config-dir, where as 'openstack overcloud export' looks for a subfolder.

tags: added: train-backport-potential ussuri-backport-potential
Revision history for this message
John Fulton (jfulton-org) wrote :

Here's the history and an explanation of what I ran into with this bug report:

A. When export was introduced it defaulted to /var/lib/mistral/<stack> [1]

B. I am running config-download manually [2] so I run the following to do an export:

   openstack overcloud config download --name $STACK --config-dir config-download
   openstack overcloud export --stack $STACK --config-download-dir config-download

C. The above worked prior to the mistral removal and will reproduce what I'm reporting

B. Export was updated during the migration from mistral to use the new default location
   so '/var/lib/mistral/<stack>' became '$HOME/config-download/<stack>' [3]

So the command in step B used to work and now it doesn't. I'm not sure if that's a bug
though. Is there a variation of the command in step B I can run which doesn't have this
issue and which doesn't require to do the workaround from comment #1?

I think the next step is to determine if people encounter this issue when not running
config-download manually. If the automatic config-download which happens when you do
not pass --stack-only fails export in the same way, then I think this is a bug for DCN
users. I'll test that next.

[1] https://github.com/openstack/python-tripleoclient/commit/975f7717029b56c7a2e6113d82e166e7c63fcba2#diff-f65025c157b1193fd86c95a092eb1000R54

[2] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/ansible_config_download.html#manual-config-download

[3] https://github.com/openstack/python-tripleoclient/commit/191438f7435aa19e9656a2052bbb6fa5ce7b6b32#diff-f65025c157b1193fd86c95a092eb1000

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/737020

Changed in tripleo:
assignee: nobody → Rabi Mishra (rabi)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ansible (master)

Reviewed: https://review.opendev.org/737020
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=4000770decfc6bef5e03e33a22cad0be9b2c5797
Submitter: Zuul
Branch: master

commit 4000770decfc6bef5e03e33a22cad0be9b2c5797
Author: Rabi Mishra <email address hidden>
Date: Fri Jun 19 20:18:12 2020 +0530

    Use folder with plan name with config download export

    To be consistent with auto config-download and overcloud export
    create subfloder with plan/stack name.

    Change-Id: I08b447379ae6e267f8dff11e6e703a991918683c
    Closes-Bug: #1884246

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/737458

Revision history for this message
John Fulton (jfulton-org) wrote :

With this patch the following works:

(undercloud) [CentOS-8.2 - stack@undercloud ~]$ openstack overcloud export --stack $STACK --output-file ~/${STACK}-export.yaml --config-download-dir ~/config-download/
Stack information exported to /home/stack/control-plane-export.yaml.
(undercloud) [CentOS-8.2 - stack@undercloud ~]$

~/config-download/ was created by running 'openstack overcloud deploy ...' without --stack-only

So far so good. However, if I don't pass --config-download-dir then it fails because it can't find this path (note the use of the stack name twice).

 /home/stack/config-download/control-plane/control-plane/group_vars/overcloud.json

It should instead be looking for the file with the stack name specified only once:

(undercloud) [CentOS-8.2 - stack@undercloud ~]$ ls /home/stack/config-download/control-plane/group_vars/
ComputeHCI Controller overcloud.json
(undercloud) [CentOS-8.2 - stack@undercloud ~]$

Here's the error:

(undercloud) [CentOS-8.2 - stack@undercloud ~]$ openstack overcloud export \
> --stack $STACK \
> --output-file ~/${STACK}-export.yaml
Exception occured while running the command
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_export.py", line 94, in take_action
    clients.orchestration, stack, False, config_download_dir))
  File "/usr/lib/python3.6/site-packages/tripleoclient/export.py", line 95, in export_stack
    with open(file, 'r') as ff:
FileNotFoundError: [Errno 2] No such file or directory: '/home/stack/config-download/control-plane/control-plane/group_vars/overcloud.json'
[Errno 2] No such file or directory: '/home/stack/config-download/control-plane/control-plane/group_vars/overcloud.json'
(undercloud) [CentOS-8.2 - stack@undercloud ~]$

Here's proof that I have the patch installed:

(undercloud) [CentOS-8.2 - stack@undercloud ~]$ grep -n -A 3 work_dir /usr/share/ansible/tripleo-playbooks/cli-config-download-export.yaml
38: - name: set work_dir
39- set_fact:
40: work_dir: "{{ config_dir ~ '/' ~ plan }}"
41-
42- tasks:
43-
44: - name: Clean work_dir
45- file:
46: path: "{{ work_dir }}"
47- state: absent
48- when: not preserve_config | bool
49-
--
52: path: "{{ work_dir }}"
53- state: directory
54-
55- - name: Get config
--
72: dest: "{{ work_dir }}"
73- remote_src: true
(undercloud) [CentOS-8.2 - stack@undercloud ~]$

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/737671

Revision history for this message
Rabi Mishra (rabi) wrote :

That's a different issue (inconsistency) on how 'overcloud export' handles default config_download_dir vs the one provided in the cli option.

https://review.opendev.org/737671 should fix that.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (master)

Reviewed: https://review.opendev.org/737671
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=e4d9bb17be805ae7bbb844f65c9f28d4d22b44e1
Submitter: Zuul
Branch: master

commit e4d9bb17be805ae7bbb844f65c9f28d4d22b44e1
Author: Rabi Mishra <email address hidden>
Date: Wed Jun 24 08:21:58 2020 +0530

    Don't use stack name in default config_download_dir

    We create a subfolder with the plan/stack name under the
    config_download_dir to store the config-downloaded artifacts.
    Fix 'overcloud export' as users are not expected to specify
    the path with the plan/stack name.

    Change-Id: I994353a6d4aa6b72e9fecc5f61bf5314a5a94963
    Related-Bug: #1884246

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/739215

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to python-tripleoclient (stable/ussuri)

Reviewed: https://review.opendev.org/739215
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=8211384480195087af1a75189de88a755a77a09c
Submitter: Zuul
Branch: stable/ussuri

commit 8211384480195087af1a75189de88a755a77a09c
Author: Rabi Mishra <email address hidden>
Date: Wed Jun 24 08:21:58 2020 +0530

    Don't use stack name in default config_download_dir

    We create a subfolder with the plan/stack name under the
    config_download_dir to store the config-downloaded artifacts.
    Fix 'overcloud export' as users are not expected to specify
    the path with the plan/stack name.

    Change-Id: I994353a6d4aa6b72e9fecc5f61bf5314a5a94963
    Related-Bug: #1884246
    (cherry picked from commit e4d9bb17be805ae7bbb844f65c9f28d4d22b44e1)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (stable/ussuri)

Reviewed: https://review.opendev.org/737458
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=46c9aa552f7ff7ce392d065d0da2c36bf4037ee8
Submitter: Zuul
Branch: stable/ussuri

commit 46c9aa552f7ff7ce392d065d0da2c36bf4037ee8
Author: Rabi Mishra <email address hidden>
Date: Tue Jun 23 14:15:32 2020 +0530

    [stable-only] Use folder with plan name for config download export

    To be consistent with auto config-download and overcloud export
    create subfloder with plan/stack name.

    Change-Id: I08b447379ae6e267f8dff11e6e703a991918683c
    Closes-Bug: #1884246

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-tripleoclient (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/741141

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to python-tripleoclient (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/741151

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-tripleoclient (stable/train)

Change abandoned by wes hayutin (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/741141
Reason: gate failure.. resetting

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-tripleoclient (stable/train)

Reviewed: https://review.opendev.org/741141
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=309df8afbe51830ccaf0af7cf39ee656d61fb828
Submitter: Zuul
Branch: stable/train

commit 309df8afbe51830ccaf0af7cf39ee656d61fb828
Author: Rabi Mishra <email address hidden>
Date: Tue Jun 23 14:15:32 2020 +0530

    [stable-only] Use folder with plan name for config download export

    To be consistent with auto config-download and overcloud export
    create subfloder with plan/stack name.

    Change-Id: I08b447379ae6e267f8dff11e6e703a991918683c
    Closes-Bug: #1884246
    (cherry picked from commit 46c9aa552f7ff7ce392d065d0da2c36bf4037ee8)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-tripleoclient (stable/train)

Change abandoned by Rabi Mishra (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/741151
Reason: Not required for stable/train

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/python-tripleoclient 12.4.0

This issue was fixed in the openstack/python-tripleoclient 12.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers