container-puppet containers fail to restart after upload-artifacts

Bug #1940571 reported by Damien Ciabrini
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Damien Ciabrini

Bug Description

In Master, during stack update, the services' config are regenerated by (re)running all container-puppet-{service}. If those containers were created prior to the stack update, they are reused and restarted during the stack update.

Re-running existing container-puppet-{service} is not idempotent, because the very first thing done in those containers is to copy the state of /etc/puppet from the host into the container's /etc. So once the containers are restarted they have an unexpected initial state.

The lack of idempotency is problematic when using upload-puppet-modules. The injection of a puppet module replaces the symlink present under /etc/puppet/modules/ with a real directory. For instance if injecting the tripleo module, one will end up with the following content on the host:

   # ls -l /etc/puppet/modules/
   [...]
   lrwxrwxrwx. 1 root root 42 Aug 19 15:19 tomcat -> /usr/share/openstack-puppet/modules/tomcat
   drwxr-xr-x. 10 heat-admin heat-admin 4096 Aug 19 15:13 tripleo
   lrwxrwxrwx. 1 root root 43 Aug 19 15:19 vcsrepo -> /usr/share/openstack-puppet/modules/vcsrepo
   [...]

As a side effect, when restarting container-puppet-{service}, the script container-puppet.sh will fail to copy the puppet content from the host to the container:

   cp: cannot overwrite non-directory '/etc/puppet/modules/tripleo' with directory '/tmp/puppet-etc/modules/tripleo'

This is because the previous puppet content is still present in the container when it is restarted, and since originally /etc/puppet/modules/tripleo was a symlink, it can't be replaced with a real directory.

Changed in tripleo:
status: New → Triaged
milestone: none → xena-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/805301
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/2e4361c878556979569a886db69b97bacf00fa29
Submitter: "Zuul (22348)"
Branch: master

commit 2e4361c878556979569a886db69b97bacf00fa29
Author: Damien Ciabrini <email address hidden>
Date: Thu Aug 19 22:07:31 2021 +0200

    container-puppet.sh idempotency on restart

    When restarting a container-puppet-{service} container,
    make sure to remove /etc/puppet from its storage,
    otherwise the config generation might fail, for example
    when using upload-puppet-module with a stack update.

    Change-Id: I1ff3363259397bba60f5ec09086cb7de34e7af8b
    Closes-Bug: #1940571

Changed in tripleo:
status: In Progress → Fix Released
tags: added: train-backport-potential wallaby-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 15.1.0

This issue was fixed in the openstack/tripleo-heat-templates 15.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/864045

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/864045
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/6b73349821f9a298ddea02bce53749fa23fdb634
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 6b73349821f9a298ddea02bce53749fa23fdb634
Author: Damien Ciabrini <email address hidden>
Date: Thu Aug 19 22:07:31 2021 +0200

    container-puppet.sh idempotency on restart

    When restarting a container-puppet-{service} container,
    make sure to remove /etc/puppet from its storage,
    otherwise the config generation might fail, for example
    when using upload-puppet-module with a stack update.

    Change-Id: I1ff3363259397bba60f5ec09086cb7de34e7af8b
    Closes-Bug: #1940571
    (cherry picked from commit 2e4361c878556979569a886db69b97bacf00fa29)

tags: added: in-stable-wallaby
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.