post-deployment config changes don't restart services

[Corey Bryant]

After a deployment of the charm, config changes don't cause services to restart. I confirmed this occurs on stable/18.02 as well as master.

The execution path goes through the render_config_restart_on_changed() 'seeded' path which only results in a service start (not a restart). This turns out to be a no-op since services are already started.

Note that for >= bionic, the bootstrap node runs mysql@bootstrap and all other nodes run mysql.

[Corey Bryant (corey.bryant)]

This doesn't appear to be a regression. I didn't test but the code seems to have the same behavior back to stable/16.04, which is the earliest branch in the charm's repo.

[David Ames (thedac)]

This bug has been present since [0]. Note specifically [1].

Unfortunately, the obvious fix, forcing a restart on any configuration file change, leads to not just restart collisions but also wsrep_cluster_address with nodes which are not not yet ready. All ending in tears.

I am wide open for suggestions.

[0] https://github.com/openstack/charm-percona-cluster/commit/1373073fd2ae6cfc4d1fb6c6a2c0e1b9f9550f04
[1] https://github.com/openstack/charm-percona-cluster/blob/master/hooks/percona_hooks.py#L265

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/555494

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/555494
Committed: https://git.openstack.org/cgit/openstack/charm-percona-cluster/commit/?id=801c2e78294f7af0a058aca036b6d066e3f53a98
Submitter: Zuul
Branch: master

commit 801c2e78294f7af0a058aca036b6d066e3f53a98
Author: David Ames <email address hidden>
Date: Mon Mar 19 11:25:56 2018 -0700

    Redesign cluster buildup process

    In order to fix bug#1756928 the whole cluster buildup process needed to
    be redesigned. The assumptions about what is_bootstrapped and clustered
    meant and when to restart on configuration changed needed to be
    re-evaluated.

    The timing of restarts needed to be protected to avoid collisions.
    Only bootstrapped hosts should go in to the
    wsrep_cluster_address=gcomm:// setting. Adding or removing units should
    be handled gracefully. Starting with a single unit and expanding to a
    cluster must work.

    This change guarantees mysqld is restarted when the configuration file
    changes and meets all the above requirements. As a consequence of the redesign,
    the workload status now more accurately reflects the state of the unit.

    Charm-helpers sync to bring in distributed_wait fix.

    Closes-Bug: #1756308
    Closes-Bug: #1756928
    Change-Id: I0742e6889b32201806cec6a0b5835e11a8027567