containers cannot replace /etc/hostname causing service failures on startup

[Alex Schultz]

If /etc/hostname is grabbed during the container configuration process, the container startup will fail with something like:

INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/mtab to /etc/mtab
INFO:__main__:Deleting /etc/hostname
ERROR:__main__:Unexpected error:
Traceback (most recent call last):
  File "/usr/local/bin/kolla_set_configs", line 411, in main
    execute_config_strategy(config)
  File "/usr/local/bin/kolla_set_configs", line 377, in execute_config_strategy
    copy_config(config)
  File "/usr/local/bin/kolla_set_configs", line 306, in copy_config
    config_file.copy()
  File "/usr/local/bin/kolla_set_configs", line 150, in copy
    self._merge_directories(source, dest)
  File "/usr/local/bin/kolla_set_configs", line 97, in _merge_directories
    os.path.join(dest, to_copy))
  File "/usr/local/bin/kolla_set_configs", line 99, in _merge_directories
    self._copy_file(source, dest)
  File "/usr/local/bin/kolla_set_configs", line 75, in _copy_file
    self._delete_path(dest)
  File "/usr/local/bin/kolla_set_configs", line 108, in _delete_path
    os.remove(path)
OSError: [Errno 16] Device or resource busy: '/etc/hostname'

This stems from the fact that /etc/hostname is a special file in docker (and podman) which cannot be removed. The content can be replaced but it cannot be removed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/738490

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/738490
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=777920dfba58fa7b26c063fa5d6df0203009d12c
Submitter: Zuul
Branch: master

commit 777920dfba58fa7b26c063fa5d6df0203009d12c
Author: Alex Schultz <email address hidden>
Date: Mon Jun 29 12:50:43 2020 -0600

    Exclude /etc/hostname

    Similar to the previous patch where we excluded /etc/hosts, it turns out
    that /etc/hostname is also a special file that cannot be synced in via
    kolla because they are bindmounted internally.

    https://github.com/containers/libpod/blob/6ac009d5304127f6758be157c7302646e5a481e5/libpod/container_internal_linux.go#L1236

    Additionally this change includes cleanup for end users who may have hit
    LP#1860607 and allows folks who have hit this to just simply rerun the
    deployment and not have to manually cleanup the synced files.

    Change-Id: I84287d9c1ef110511c66743f9a3a22709077b0cb
    Closes-Bug: #1885619

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/739496

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/739497

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/739496
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=7ab3de5899440c4ce98a7ea8c7344bc5f13cafdb
Submitter: Zuul
Branch: stable/ussuri

commit 7ab3de5899440c4ce98a7ea8c7344bc5f13cafdb
Author: Alex Schultz <email address hidden>
Date: Mon Jun 29 12:50:43 2020 -0600

    Exclude /etc/hostname

    Similar to the previous patch where we excluded /etc/hosts, it turns out
    that /etc/hostname is also a special file that cannot be synced in via
    kolla because they are bindmounted internally.

    https://github.com/containers/libpod/blob/6ac009d5304127f6758be157c7302646e5a481e5/libpod/container_internal_linux.go#L1236

    Additionally this change includes cleanup for end users who may have hit
    LP#1860607 and allows folks who have hit this to just simply rerun the
    deployment and not have to manually cleanup the synced files.

    Change-Id: I84287d9c1ef110511c66743f9a3a22709077b0cb
    Closes-Bug: #1885619
    (cherry picked from commit 777920dfba58fa7b26c063fa5d6df0203009d12c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/739497
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=39945d5630c0d83a8b7fafd6b45e945c3e89c800
Submitter: Zuul
Branch: stable/train

commit 39945d5630c0d83a8b7fafd6b45e945c3e89c800
Author: Alex Schultz <email address hidden>
Date: Mon Jun 29 12:50:43 2020 -0600

    Exclude /etc/hostname

    Similar to the previous patch where we excluded /etc/hosts, it turns out
    that /etc/hostname is also a special file that cannot be synced in via
    kolla because they are bindmounted internally.

    https://github.com/containers/libpod/blob/6ac009d5304127f6758be157c7302646e5a481e5/libpod/container_internal_linux.go#L1236

    Additionally this change includes cleanup for end users who may have hit
    LP#1860607 and allows folks who have hit this to just simply rerun the
    deployment and not have to manually cleanup the synced files.

    Change-Id: I84287d9c1ef110511c66743f9a3a22709077b0cb
    Closes-Bug: #1885619
    (cherry picked from commit 777920dfba58fa7b26c063fa5d6df0203009d12c)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/745716

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/745716
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9801dc7461cbd6cbd73868e72e74d21d586c6708
Submitter: Zuul
Branch: stable/queens

commit 9801dc7461cbd6cbd73868e72e74d21d586c6708
Author: Alex Schultz <email address hidden>
Date: Mon Jun 29 12:50:43 2020 -0600

    Exclude /etc/hostname

    Note: this is an unclean backport and we had to create
    puppet_generated_path var to make it simpler (and avoid path
    repetition).

    Similar to the previous patch where we excluded /etc/hosts, it turns out
    that /etc/hostname is also a special file that cannot be synced in via
    kolla because they are bindmounted internally.

    https://github.com/containers/libpod/blob/6ac009d5304127f6758be157c7302646e5a481e5/libpod/container_internal_linux.go#L1236

    Additionally this change includes cleanup for end users who may have hit
    LP#1860607 and allows folks who have hit this to just simply rerun the
    deployment and not have to manually cleanup the synced files.

    Change-Id: I84287d9c1ef110511c66743f9a3a22709077b0cb
    Closes-Bug: #1885619
    (cherry picked from commit 777920dfba58fa7b26c063fa5d6df0203009d12c)
    (cherry picked from commit 39945d5630c0d83a8b7fafd6b45e945c3e89c800)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates queens-eol release.