Swift config changes are not applied on overcloud update

Bug #1802066 reported by Christian Schwede
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Christian Schwede

Bug Description

Noticed this when adding a new disk to an existing deployment.

For example, adding disk vdb to an deployment that is currently using "d1" as the only device.

There are two issues after the deployment finished:

1. The rings in /var/lib/config-data/puppet-generated/swift/etc/swift are not updated (rings in /var/lib/config-data/swift_ringbuilder are updated, but not copied over)
2. Permissions on /srv/node are wrong

The issue here is that the one-time containers swift_setup_srv and swift_copy_rings are not run again.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.openstack.org/616116

tags: added: pike-backport-potential queens-backport-potential rocky-backport-potential
Changed in tripleo:
milestone: stein-2 → stein-3
Changed in tripleo:
assignee: Christian Schwede (cschwede) → Alex Schultz (alex-schultz)
Changed in tripleo:
assignee: Alex Schultz (alex-schultz) → Christian Schwede (cschwede)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.openstack.org/616116
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b49629f085b5f04be999e0d5697a77e3791f7dcc
Submitter: Zuul
Branch: master

commit b49629f085b5f04be999e0d5697a77e3791f7dcc
Author: Christian Schwede <email address hidden>
Date: Wed Nov 7 08:53:48 2018 +0100

    Do not ignore Swift ring changes to trigger container restart

    Swift containers need to restart if the rings change. In
    non-containerized environments this is not required, because the Swift
    processes will reload the rings on any changes. However, this does not
    work within containers, thus a restart is required.

    This also restarts swift_copy_rings and swift_setup_srv container. This
    will copy the updated ring files and ensure new storage mount points are
    using the right permissions.

    Closes-Bug: 1802066
    Related-Bug: 1786065
    Change-Id: Ie2b9f003dc34f2f02a45293d06d6a40c8d5ed8ff

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 10.4.0

This issue was fixed in the openstack/tripleo-heat-templates 10.4.0 release.

tags: added: idempotency
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/713415

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (master)

Change abandoned by Bogdan Dobrelya (bogdando) (<email address hidden>) on branch: master
Review: https://review.opendev.org/713415

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Bogdan Dobrelya (bogdando) (<email address hidden>) on branch: master
Review: https://review.opendev.org/713415
Reason: thanks for testing this!

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/713415
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9bc664090741d43fd49857230238fc345bf8d0c9
Submitter: Zuul
Branch: master

commit 9bc664090741d43fd49857230238fc345bf8d0c9
Author: Bogdan Dobrelya <email address hidden>
Date: Tue Mar 17 12:45:15 2020 +0100

    Fix swift containers idempotency

    The change https://review.opendev.org/#/c/616116 unwinded the swift
    part of the https://review.opendev.org/#/c/590008/ changes. So the
    contents of the /var/lib/config-data/swift_ringbuilder config volume
    used to be managed by container-puppet tool. That made swift containers
    always restarted because the puppet-generated rings are always changing
    on each deployment/update execution.

    Restore that unwinded change back and exclude swift rings from the
    management of container-puppet tooling. Instead make init containers
    swift_copy_rings and swift_setup_srv to be always executed (takes
    the same approach as in https://review.opendev.org/#/c/564798/).

    That also fixes the issue with swift_copy_rings seems never been
    executed - at least there is no traces of it in CI jobs logs for swift
    init containers.

    Change-Id: I23b469057e4c47c42601beb166f815ee71147c14
    Closes-Bug: #1867765
    Related-Bug: #1802066
    Related-Bug: #1786065
    Signed-off-by: Bogdan Dobrelya <email address hidden>

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.