Installation of undercloud with ssl may fails as keepalived may not be started before haproxy

Bug #1638029 reported by Sofer Athlan-Guyot
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Sofer Athlan-Guyot

Bug Description

Hi,

installing the undercloud with ssl enable may sometimes fails if the keepalived service is not started before the haproxy service.

The final error is:

    Could not evaluate: Execution of '/bin/openstack token issue --format value' returned 1: Unable to establish connection to https://192.168.0.2:13000/v3/auth/tokens (tried 22, for a total of 170 seconds)

during

    puppet apply --debug --detailed-exitcodes /etc/puppet/manifests/puppet-stack-config.pp

(openstack undercloud install)

The first time the error has been is in this bz : https://bugzilla.redhat.com/show_bug.cgi?id=1388283 for osp9 ssl installation.

Changed in tripleo:
assignee: nobody → Sofer Athlan-Guyot (sofer-athlan-guyot)
status: New → In Progress
Changed in tripleo:
milestone: none → ocata-1
tags: removed: mitaka-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/391873
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=aa82e175b5276729385f9b938f34a291b98640f5
Submitter: Jenkins
Branch: master

commit aa82e175b5276729385f9b938f34a291b98640f5
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Oct 31 15:53:13 2016 +0100

    Make sure keepalived is restarted before haproxy.

    When using SSL setup for undercloud, the admin and public vip required
    for ssl binding by haproxy are created by keepalived.

    This makes sure that keepalived is started before haproxy and thus that
    the interfaces are indeed present.

    This patch also ensures this is happening for overcloud ssl
    configuration. The case where another load-balancing technology other
    than haproxy is used is not covered.

    Closes-Bug: #1638029

    Change-Id: I98cb0dcd7f389a1dd38ec8324429bfef4979aa66

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/393361

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (stable/newton)

Reviewed: https://review.openstack.org/393361
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=2da35636b0218b5ebe365f531faee384bdd8c37f
Submitter: Jenkins
Branch: stable/newton

commit 2da35636b0218b5ebe365f531faee384bdd8c37f
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Oct 31 15:53:13 2016 +0100

    Make sure keepalived is restarted before haproxy.

    When using SSL setup for undercloud, the admin and public vip required
    for ssl binding by haproxy are created by keepalived.

    This makes sure that keepalived is started before haproxy and thus that
    the interfaces are indeed present.

    This patch also ensures this is happening for overcloud ssl
    configuration. The case where another load-balancing technology other
    than haproxy is used is not covered.

    Closes-Bug: #1638029

    Change-Id: I98cb0dcd7f389a1dd38ec8324429bfef4979aa66
    (cherry picked from commit aa82e175b5276729385f9b938f34a291b98640f5)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/395053

Revision history for this message
Marios Andreou (marios-b) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/395053
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=b9dcee028795bb1132176df7ef3f435039cc20cb
Submitter: Jenkins
Branch: master

commit b9dcee028795bb1132176df7ef3f435039cc20cb
Author: Sofer Athlan-Guyot <email address hidden>
Date: Tue Nov 8 16:44:26 2016 +0100

    Ensure keepalived is restarted when necessary.

    If os-collect-config/config.json is updated before an upgrade/update,
    then the os-net-config run will automatically erase the keepalived
    managed ips.

    This is a hackish way to ensure that keepalived is restarted during the
    next phase in order to have the ip recreated.

    It basically adds a comment line to the keepalived.conf file (making it
    different than the puppet one) if it's there. This will force a puppet
    restart of the keepalive service puting the ips back on the undercloud.

    Change-Id: I56b706ff44ba31aa87a63f870940831ce02a6e77
    Closes-Bug: #1640213
    Related-Bug: #1638029

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (stable/newton)

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/396731

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (stable/newton)

Reviewed: https://review.openstack.org/396731
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=3e586d0fb22cfd9c08e8f4663ab9be140af83d7b
Submitter: Jenkins
Branch: stable/newton

commit 3e586d0fb22cfd9c08e8f4663ab9be140af83d7b
Author: Sofer Athlan-Guyot <email address hidden>
Date: Tue Nov 8 16:44:26 2016 +0100

    Ensure keepalived is restarted when necessary.

    If os-collect-config/config.json is updated before an upgrade/update,
    then the os-net-config run will automatically erase the keepalived
    managed ips.

    This is a hackish way to ensure that keepalived is restarted during the
    next phase in order to have the ip recreated.

    It basically adds a comment line to the keepalived.conf file (making it
    different than the puppet one) if it's there. This will force a puppet
    restart of the keepalive service puting the ips back on the undercloud.

    Change-Id: I56b706ff44ba31aa87a63f870940831ce02a6e77
    Closes-Bug: #1640213
    Related-Bug: #1638029
    (cherry picked from commit b9dcee028795bb1132176df7ef3f435039cc20cb)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 5.4.0

This issue was fixed in the openstack/puppet-tripleo 5.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 6.0.0

This issue was fixed in the openstack/puppet-tripleo 6.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 5.4.0

This issue was fixed in the openstack/puppet-tripleo 5.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.