kolla_set_configs --check fails for source directories

Bug #1890567 reported by Mark Goddard on 2020-08-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Medium
Radosław Piliszek
Train
Medium
Unassigned
Ussuri
Medium
Unassigned
Victoria
Medium
Radosław Piliszek

Bug Description

There are two issues with kolla_set_configs --check:

1. We calculate the destination path incorrectly when comparing a file
   in a directory, due to passing arguments to os.path.relpath in the
   wrong order
2. For directories that have not changed, we also attempt to compare
   them as files, which fails when they are open()ed.

This affects the fluentd container, which specifies directories as the
source, without using a glob:

{
   "source": "/var/lib/kolla/config_files/output",
    "dest": "/etc/td-agent/output",
    "owner": "td-agent",
    "perm": "0600"
}

This results in an error like this:

INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
ERROR:__main__:Unexpected error:
Traceback (most recent call last):
  File "/usr/local/bin/kolla_set_configs", line 423, in main
    execute_config_check(config)
  File "/usr/local/bin/kolla_set_configs", line 409, in execute_config_check
    config_file.check()
  File "/usr/local/bin/kolla_set_configs", line 228, in check
    if os.path.isdir(source) and not self._cmp_dir(source, dest):
  File "/usr/local/bin/kolla_set_configs", line 209, in _cmp_dir
    if not self._cmp_file(full_path, dest_full_path):
  File "/usr/local/bin/kolla_set_configs", line 167, in _cmp_file
    with open(source) as f1, open(dest) as f2:
IsADirectoryError: [Errno 21] Is a directory: '/etc/td-agent/input/..'

Fix proposed to branch: master
Review: https://review.opendev.org/745105

Changed in kolla:
assignee: nobody → Mark Goddard (mgoddard)
status: New → In Progress
Mark Goddard (mgoddard) on 2020-08-06
Changed in kolla:
importance: Undecided → Medium
Changed in kolla:
assignee: Mark Goddard (mgoddard) → Radosław Piliszek (yoctozepto)

Reviewed: https://review.opendev.org/745105
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c5320eb22386c34d83435fa0b9aa03b178996b52
Submitter: Zuul
Branch: master

commit c5320eb22386c34d83435fa0b9aa03b178996b52
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822

Changed in kolla:
status: In Progress → Fix Released

Reviewed: https://review.opendev.org/751728
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c6835b12a6f1e8f715aa5c69383a8d1836551496
Submitter: Zuul
Branch: stable/stein

commit c6835b12a6f1e8f715aa5c69383a8d1836551496
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

tags: added: in-stable-stein

Reviewed: https://review.opendev.org/751726
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c191e6fbd7ebf0e6f455dc8cfc34ecda64e33bc1
Submitter: Zuul
Branch: stable/ussuri

commit c191e6fbd7ebf0e6f455dc8cfc34ecda64e33bc1
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

tags: added: in-stable-ussuri

Reviewed: https://review.opendev.org/751727
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=797ecd6ddbadcc7a726ce1611c210667565d8a91
Submitter: Zuul
Branch: stable/train

commit 797ecd6ddbadcc7a726ce1611c210667565d8a91
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

tags: added: in-stable-train

This issue was fixed in the openstack/kolla 8.0.5 release.

This issue was fixed in the openstack/kolla 10.2.0 release.

This issue was fixed in the openstack/kolla 9.3.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers