Default Swift ring deployment by TripleO are error-prone

[Christian Schwede]

There are a few problems today when deploying an overcloud with Swift using TripleO.

- only a single device per node is deployed
- this device is just a directory; locally available disks aren't used
- mount_check is disabled; this results in writing data to the root device
- adding a new node or replacing an existing one will result in inconsistent rings, because rings are created locally on each node and nodes don't know the "history"
- manually managed rings are overwritten on each deployment
- the ringbuilder doesn't know how to handle storage policies, for example to support Erasure Coding

A discussion how to improve this has been started on openstack-dev:
http://lists.openstack.org/pipermail/openstack-dev/2016-August/100720.html

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/358643

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/406633

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christian Schwede (<email address hidden>) on branch: master
Review: https://review.openstack.org/358643
Reason: Abandoning this one. A better workflow is to build rings after the nodes are already started; by then we know the hostnames and can use the existing ones.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/413230

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/414458

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/414460

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christian Schwede (<email address hidden>) on branch: master
Review: https://review.openstack.org/414458
Reason: Squashed into parent patch.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christian Schwede (<email address hidden>) on branch: master
Review: https://review.openstack.org/406633
Reason: Abandoned in favor of https://review.openstack.org/414460

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christian Schwede (<email address hidden>) on branch: master
Review: https://review.openstack.org/413230
Reason: Abandoned in favor of https://review.openstack.org/414460

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/413229
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=067049594a9e2a770c7851feb134fdcbcce8e6ce
Submitter: Jenkins
Branch: master

commit 067049594a9e2a770c7851feb134fdcbcce8e6ce
Author: Christian Schwede <email address hidden>
Date: Tue Dec 20 16:50:19 2016 +0000

    Use Mistral to create Swift temporary URLs

    This patch consists of two parts.

    A new Mistral action has been added to create signed temporary URLs.
    The method to generate temporary URLs isnt' exposed by
    python-swiftclient, therefore adding that method using it's own action.
    It also sets the required metadata with a random key if not yet
    existing.

    The deployment workbook has been updated to create two temporary URLs
    (one PUT, one GET) that can be used during the deployment for up- and
    downloading Swift rings to the undercloud node.

    Related-Bug: 1609421
    Change-Id: Ic3da38cffdd993c768bdb137c17d625dff1aa372

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/414460
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b323f8a16035549d84cdec4718380bde3d23d6c3
Submitter: Jenkins
Branch: master

commit b323f8a16035549d84cdec4718380bde3d23d6c3
Author: Christian Schwede <email address hidden>
Date: Fri Dec 23 08:27:10 2016 +0100

    Automatically backup and restore Swift rings from the undercloud

    Swift rings created or updated on the overcloud nodes will now be
    stored on the undercloud at the end of the deployment. An
    additional consistency check is executed before storing them,
    ensuring all rings within the cluster are identical.

    These rings will be retrieved (before Puppet runs) by every node
    when an UPDATE is executed, and by doing this will be in a
    consistent state across the cluster.

    This makes it possible to add, remove or replace nodes in an
    existing cluster without manual operator interaction.

    Closes-Bug: 1609421
    Depends-On: Ic3da38cffdd993c768bdb137c17d625dff1aa372
    Change-Id: I758179182265da5160c06bb95f4c6258dc0edcd6

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/435191

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/435284

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/435285

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/435284
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=c79248d50d0c18f947a5fa9ebc452ebf9b49260e
Submitter: Jenkins
Branch: stable/newton

commit c79248d50d0c18f947a5fa9ebc452ebf9b49260e
Author: Christian Schwede <email address hidden>
Date: Tue Dec 20 16:50:19 2016 +0000

    Use Mistral to create Swift temporary URLs

    This patch consists of two parts.

    A new Mistral action has been added to create signed temporary URLs.
    The method to generate temporary URLs isnt' exposed by
    python-swiftclient, therefore adding that method using it's own action.
    It also sets the required metadata with a random key if not yet
    existing.

    The deployment workbook has been updated to create two temporary URLs
    (one PUT, one GET) that can be used during the deployment for up- and
    downloading Swift rings to the undercloud node.

    Related-Bug: 1609421
    Change-Id: Ic3da38cffdd993c768bdb137c17d625dff1aa372
    (cherry picked from commit 067049594a9e2a770c7851feb134fdcbcce8e6ce)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/435191
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a334a1545db4f5411037ce8a2e397c7ac86daf32
Submitter: Jenkins
Branch: stable/ocata

commit a334a1545db4f5411037ce8a2e397c7ac86daf32
Author: Christian Schwede <email address hidden>
Date: Fri Dec 23 08:27:10 2016 +0100

    Automatically backup and restore Swift rings from the undercloud

    Swift rings created or updated on the overcloud nodes will now be
    stored on the undercloud at the end of the deployment. An
    additional consistency check is executed before storing them,
    ensuring all rings within the cluster are identical.

    These rings will be retrieved (before Puppet runs) by every node
    when an UPDATE is executed, and by doing this will be in a
    consistent state across the cluster.

    This makes it possible to add, remove or replace nodes in an
    existing cluster without manual operator interaction.

    Closes-Bug: 1609421
    Depends-On: Ic3da38cffdd993c768bdb137c17d625dff1aa372
    Change-Id: I758179182265da5160c06bb95f4c6258dc0edcd6
    (cherry picked from commit b323f8a16035549d84cdec4718380bde3d23d6c3)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 6.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/435285
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=af74df75c6ba67e88b3bf34d10c6ebde91c9082f
Submitter: Jenkins
Branch: stable/newton

commit af74df75c6ba67e88b3bf34d10c6ebde91c9082f
Author: Christian Schwede <email address hidden>
Date: Fri Dec 23 08:27:10 2016 +0100

    Automatically backup and restore Swift rings from the undercloud

    Swift rings created or updated on the overcloud nodes will now be
    stored on the undercloud at the end of the deployment. An
    additional consistency check is executed before storing them,
    ensuring all rings within the cluster are identical.

    These rings will be retrieved (before Puppet runs) by every node
    when an UPDATE is executed, and by doing this will be in a
    consistent state across the cluster.

    This makes it possible to add, remove or replace nodes in an
    existing cluster without manual operator interaction.

    Closes-Bug: 1609421
    Depends-On: Ic3da38cffdd993c768bdb137c17d625dff1aa372
    Change-Id: I758179182265da5160c06bb95f4c6258dc0edcd6
    (cherry picked from commit b323f8a16035549d84cdec4718380bde3d23d6c3)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.3.0 release.