all slave keepalived down after upgrade

Bug #1836368 reported by Radosław Piliszek
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
Medium
Mark Goddard
Queens
Fix Committed
Medium
Mark Goddard
Rocky
Fix Committed
Medium
Mark Goddard
Stein
Fix Committed
Medium
Mark Goddard
Train
Fix Released
Medium
Mark Goddard

Bug Description

Stein, master (possibly older)

We do "Stopping all slave keepalived containers" on upgrade but never start slaves again.
If there is no config change, then all atm slaves will remain down until started manually by op.

Revision history for this message
Mark Goddard (mgoddard) wrote :

The 'Check haproxy containers' task should cover this case, since it triggers the handler.

Revision history for this message
Radosław Piliszek (yoctozepto) wrote :

Yup, this affects partial/same-to-same upgrades.

Revision history for this message
Mark Goddard (mgoddard) wrote :

Ah. In which case should "Stopping all slave keepalived containers" itself be a handler?

Revision history for this message
Radosław Piliszek (yoctozepto) wrote :

I think it should at least notify restart.

Mark Goddard (mgoddard)
Changed in kolla-ansible:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/683980

Changed in kolla-ansible:
assignee: nobody → Mark Goddard (mgoddard)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/683980
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=6f05f1b8441a4a3097c0f596e8ea0fdf3d6240ee
Submitter: Zuul
Branch: master

commit 6f05f1b8441a4a3097c0f596e8ea0fdf3d6240ee
Author: Mark Goddard <email address hidden>
Date: Mon Sep 23 15:25:54 2019 +0100

    Ensure keepalived is restarted during upgrade

    During upgrade, we stop all slave keepalived containers. However, if the
    keepalived container configuration has not changed, we never restart
    them.

    This change fixes the issue by notifying the restart handler when the
    containers are stopped.

    Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
    Closes-Bug: #1836368

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/684250

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/684251

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/684252

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/rocky)

Reviewed: https://review.opendev.org/684251
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=57e9679f011ec9322705f0aa73f4cfa7913dd218
Submitter: Zuul
Branch: stable/rocky

commit 57e9679f011ec9322705f0aa73f4cfa7913dd218
Author: Mark Goddard <email address hidden>
Date: Mon Sep 23 15:25:54 2019 +0100

    Ensure keepalived is restarted during upgrade

    During upgrade, we stop all slave keepalived containers. However, if the
    keepalived container configuration has not changed, we never restart
    them.

    This change fixes the issue by notifying the restart handler when the
    containers are stopped.

    Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
    Closes-Bug: #1836368
    (cherry picked from commit 6f05f1b8441a4a3097c0f596e8ea0fdf3d6240ee)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/stein)

Reviewed: https://review.opendev.org/684250
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=32e8f53ea4ea441855d931a2a1f46d71bbb76e13
Submitter: Zuul
Branch: stable/stein

commit 32e8f53ea4ea441855d931a2a1f46d71bbb76e13
Author: Mark Goddard <email address hidden>
Date: Mon Sep 23 15:25:54 2019 +0100

    Ensure keepalived is restarted during upgrade

    During upgrade, we stop all slave keepalived containers. However, if the
    keepalived container configuration has not changed, we never restart
    them.

    This change fixes the issue by notifying the restart handler when the
    containers are stopped.

    Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
    Closes-Bug: #1836368
    (cherry picked from commit 6f05f1b8441a4a3097c0f596e8ea0fdf3d6240ee)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (stable/queens)

Reviewed: https://review.opendev.org/684252
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=3f8d918abfd2448200710cbf0999f8f8d5bf79a9
Submitter: Zuul
Branch: stable/queens

commit 3f8d918abfd2448200710cbf0999f8f8d5bf79a9
Author: Mark Goddard <email address hidden>
Date: Mon Sep 23 15:25:54 2019 +0100

    Ensure keepalived is restarted during upgrade

    During upgrade, we stop all slave keepalived containers. However, if the
    keepalived container configuration has not changed, we never restart
    them.

    This change fixes the issue by notifying the restart handler when the
    containers are stopped.

    Change-Id: Ibe094b0c14a70a0eb811182d96f045027aa02c2a
    Closes-Bug: #1836368
    (cherry picked from commit 6f05f1b8441a4a3097c0f596e8ea0fdf3d6240ee)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 6.2.3

This issue was fixed in the openstack/kolla-ansible 6.2.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 9.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 7.2.0

This issue was fixed in the openstack/kolla-ansible 7.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 8.1.0

This issue was fixed in the openstack/kolla-ansible 8.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.