memcached mongodb and apache not restarted automatically on HA deployments

[Damien Ciabrini]

Prior to Newton, in HA deployments, pacemaker would notice when a
service running on controller node is down and restart it
automatically.

Since Newton and the new HA architecture, the vast majority of services
are now managed by systemd. All Openstack services are configured to
be restarted automatically by systemd should they fail unexpected.

e.g. for /usr/lib/systemd/system/openstack-nova-conductor.service:

...
[Service]
Restart=always
...

However, non-Openstack services like memcached, apache and
mongodb have a default configuration which does not enable
automatic restart on failure.

A manual intervention is required to restart service which is
not ideal. Moreover, it may lead to service disruption if failure
eventually happens on all nodes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/531190

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/531190
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=e84da28dc310f83ef6ac8ae61ccb4cc5fe6d3c2e
Submitter: Zuul
Branch: master

commit e84da28dc310f83ef6ac8ae61ccb4cc5fe6d3c2e
Author: Damien Ciabrini <email address hidden>
Date: Thu Jan 4 15:27:45 2018 +0100

    Enable automatic restart of memcached, apache and mongodb

    Prior to Newton, HA deployments had automatic restart of non-Openstack
    services on controller nodes. From Newton onward, after we switched to
    systemd for the vast majority of services, Openstack services are
    configured for automatic restart, but memcached apache and mongodb
    lack it.

    Generate a drop-in file for the mentioned services, so they can
    restart automatically in both HA and non-HA deployments.

    Change-Id: I8576db193b358cf804e8aaeb0e05b31c635ae5ba
    Closes-Bug: #1741263

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/532738

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/532740

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/532743

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/534394

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/532738
Reason: https://review.openstack.org/#/c/534394/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/532740
Reason: we'll backport https://review.openstack.org/#/c/534394/ to ocata

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/532743
Reason: we'll backport https://review.openstack.org/#/c/534394/ to newton

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/534755

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/534951

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/534394
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=6bbee22929d7ada0fe4565179840011fa57e478d
Submitter: Zuul
Branch: stable/pike

commit 6bbee22929d7ada0fe4565179840011fa57e478d
Author: Damien Ciabrini <email address hidden>
Date: Tue Jan 16 17:37:39 2018 +0000

    Enable automatic restart of memcached, apache and mongodb

    Prior to Newton, HA deployments had automatic restart of non-Openstack
    services on controller nodes. From Newton onward, after we switched to
    systemd for the vast majority of services, Openstack services are
    configured for automatic restart, but memcached apache and mongodb
    lack it.

    Generate a drop-in file for the mentioned services, so they can
    restart automatically in both HA and non-HA deployments.

    This change is similar to the original review in master [1], but
    it is implemented without puppet-systemd, because all stable versions
    upstream (N, O, P) ship with an outdated puppet-systemd and one
    impacted version downstream (N) doesn't even ship puppet-systemd.

    [1] I8576db193b358cf804e8aaeb0e05b31c635ae5ba

    Change-Id: I7658f539db55b7416a9931822cfe2102c3388123
    Closes-Bug: #1741263

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 8.2.0 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/534951
Reason: Gate resets is causing issue to land a critical patch (538346) - I need to clear the gate until 538346 is merged. Please do not restore this patch, I'll take care of it.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/534951
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=9b68a3ad438175cf15a3897db8434fed86a19f96
Submitter: Zuul
Branch: stable/newton

commit 9b68a3ad438175cf15a3897db8434fed86a19f96
Author: Damien Ciabrini <email address hidden>
Date: Wed Jan 17 21:59:52 2018 +0100

    Enable automatic restart of memcached, apache and mongodb

    Prior to Newton, HA deployments had automatic restart of non-Openstack
    services on controller nodes. From Newton onward, after we switched to
    systemd for the vast majority of services, Openstack services are
    configured for automatic restart, but memcached apache and mongodb
    lack it.

    Generate a drop-in file for the mentioned services, so they can
    restart automatically in both HA and non-HA deployments.

    This change differs slighly from the review backported to Ocata [1]
    because in Newton, there is no tripleo-specific profile to configure
    Apache; So we generate the drop-in file in the keystone profile to
    guarantee that Apache automatic restart will be enabled on nodes where
    OpenStack API are deployed.

    [1] I7658f539db55b7416a9931822cfe2102c3388123

    Change-Id: Ic6b57ec30c53a077eb83afd5c99f5c59e6ff56d3
    Closes-Bug: #1741263
    Co-Authored-By: Michele Baldessari <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/534755
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=695f35dfaf286240916ee248248f11a99ad72b32
Submitter: Zuul
Branch: stable/ocata

commit 695f35dfaf286240916ee248248f11a99ad72b32
Author: Damien Ciabrini <email address hidden>
Date: Tue Jan 16 17:37:39 2018 +0000

    Enable automatic restart of memcached, apache and mongodb

    Prior to Newton, HA deployments had automatic restart of non-Openstack
    services on controller nodes. From Newton onward, after we switched to
    systemd for the vast majority of services, Openstack services are
    configured for automatic restart, but memcached apache and mongodb
    lack it.

    Generate a drop-in file for the mentioned services, so they can
    restart automatically in both HA and non-HA deployments.

    This change is similar to the original review in master [1], but
    it is implemented without puppet-systemd, because all stable versions
    upstream (N, O, P) ship with an outdated puppet-systemd and one
    impacted version downstream (N) doesn't even ship puppet-systemd.

    [1] I8576db193b358cf804e8aaeb0e05b31c635ae5ba

    Change-Id: I7658f539db55b7416a9931822cfe2102c3388123
    Closes-Bug: #1741263

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 6.5.9 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 7.4.9 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/puppet-tripleo 5.6.8 release.