kolla_set_configs --check fails for source directories

[Mark Goddard]

There are two issues with kolla_set_configs --check:

1. We calculate the destination path incorrectly when comparing a file
   in a directory, due to passing arguments to os.path.relpath in the
   wrong order
2. For directories that have not changed, we also attempt to compare
   them as files, which fails when they are open()ed.

This affects the fluentd container, which specifies directories as the
source, without using a glob:

{
   "source": "/var/lib/kolla/config_files/output",
    "dest": "/etc/td-agent/output",
    "owner": "td-agent",
    "perm": "0600"
}

This results in an error like this:

INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
ERROR:__main__:Unexpected error:
Traceback (most recent call last):
  File "/usr/local/bin/kolla_set_configs", line 423, in main
    execute_config_check(config)
  File "/usr/local/bin/kolla_set_configs", line 409, in execute_config_check
    config_file.check()
  File "/usr/local/bin/kolla_set_configs", line 228, in check
    if os.path.isdir(source) and not self._cmp_dir(source, dest):
  File "/usr/local/bin/kolla_set_configs", line 209, in _cmp_dir
    if not self._cmp_file(full_path, dest_full_path):
  File "/usr/local/bin/kolla_set_configs", line 167, in _cmp_file
    with open(source) as f1, open(dest) as f2:
IsADirectoryError: [Errno 21] Is a directory: '/etc/td-agent/input/..'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/745105

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/745105
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c5320eb22386c34d83435fa0b9aa03b178996b52
Submitter: Zuul
Branch: master

commit c5320eb22386c34d83435fa0b9aa03b178996b52
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/751726

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/751727

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/751728

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/751728
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c6835b12a6f1e8f715aa5c69383a8d1836551496
Submitter: Zuul
Branch: stable/stein

commit c6835b12a6f1e8f715aa5c69383a8d1836551496
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/751726
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c191e6fbd7ebf0e6f455dc8cfc34ecda64e33bc1
Submitter: Zuul
Branch: stable/ussuri

commit c191e6fbd7ebf0e6f455dc8cfc34ecda64e33bc1
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/751727
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=797ecd6ddbadcc7a726ce1611c210667565d8a91
Submitter: Zuul
Branch: stable/train

commit 797ecd6ddbadcc7a726ce1611c210667565d8a91
Author: Mark Goddard <email address hidden>
Date: Thu Aug 6 10:50:25 2020 +0100

    Fix kolla_set_configs --check with a directory

    There are several issues with kolla_set_configs --check:

    1. We calculate the destination path incorrectly when comparing a file
       in a directory, due to passing arguments to os.path.relpath in the
       wrong order
    2. For directories that have not changed, we also attempt to compare
       them as files, which fails when they are open()ed.
    3. If the config JSON does not have a config_files key, it fails with a
       KeyError.

    The first two issues affect the fluentd container, which specifies
    directories as the source, without using a glob. The third affects OVN
    containers.

    This patch fixes these issues.

    Closes-Bug: #1890567

    Change-Id: I8921befe51da4282121443849177a7ca5ebe8822
    (cherry picked from commit c5320eb22386c34d83435fa0b9aa03b178996b52)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 8.0.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 10.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 9.3.0 release.