Instances failed over while service was down

Bug #1841777 reported by Lucian Petrut
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
compute-hyperv
Fix Released
Undecided
Lucian Petrut

Bug Description

At the moment, the driver won't handle instances that were failed over while the service was down. This is due to a regression, the method doing this isn't called properly.

At the same time, we currently attempt to claim instances before the distributed lock manager is initialized. This will fail, but not always, as this is executed asynchronously.

Revision history for this message
Lucian Petrut (petrutlucian94) wrote :
Changed in compute-hyperv:
assignee: nobody → Lucian Petrut (petrutlucian94)
status: New → In Progress
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to compute-hyperv (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/679726

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to compute-hyperv (master)

Reviewed: https://review.opendev.org/679049
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=e8e2bedd76caaa80964571834bc069937f2004dd
Submitter: Zuul
Branch: master

commit e8e2bedd76caaa80964571834bc069937f2004dd
Author: Lucian Petrut <email address hidden>
Date: Wed Aug 28 15:58:59 2019 +0300

    Handle instances failed over while nova was down

    At the moment, the driver won't handle instances that were failed
    over while the service was down. This is due to a regression,
    the method doing this isn't called properly.

    Closes-Bug: #1841777

    Change-Id: I4ecf916b1d3323077646a38b9491785accb3c87c

Changed in compute-hyperv:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to compute-hyperv (master)

Reviewed: https://review.opendev.org/679726
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=20c1b2083f4e684f731d1e0593024b4685dedd7d
Submitter: Zuul
Branch: master

commit 20c1b2083f4e684f731d1e0593024b4685dedd7d
Author: Lucian Petrut <email address hidden>
Date: Tue Sep 3 11:22:14 2019 +0300

    Initialize coordination during driver init

    At the moment, the "init_host" method initializes the distributed
    lock mechanism, yet the driver may try to use distributed locks
    before that.

    For example, the method that reclaims failed over instances is
    called by the driver init method, before init_host. While we
    can fix this particular call, it's safer that we initialize the
    lock mechanism as early as possible.

    Change-Id: If361c0f754d3680726762e3126286227bea6f3f3
    Related-Bug: #1841777

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to compute-hyperv (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/680351

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to compute-hyperv (stable/stein)

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/680352

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to compute-hyperv (stable/stein)

Reviewed: https://review.opendev.org/680352
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=827bb4c9211f5407ca130d93b207dece834690be
Submitter: Zuul
Branch: stable/stein

commit 827bb4c9211f5407ca130d93b207dece834690be
Author: Lucian Petrut <email address hidden>
Date: Tue Sep 3 11:22:14 2019 +0300

    Initialize coordination during driver init

    At the moment, the "init_host" method initializes the distributed
    lock mechanism, yet the driver may try to use distributed locks
    before that.

    For example, the method that reclaims failed over instances is
    called by the driver init method, before init_host. While we
    can fix this particular call, it's safer that we initialize the
    lock mechanism as early as possible.

    Change-Id: If361c0f754d3680726762e3126286227bea6f3f3
    Related-Bug: #1841777
    (cherry picked from commit 20c1b2083f4e684f731d1e0593024b4685dedd7d)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to compute-hyperv (stable/stein)

Reviewed: https://review.opendev.org/680351
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=2142b610f3703ecb969777fa11371214570d05af
Submitter: Zuul
Branch: stable/stein

commit 2142b610f3703ecb969777fa11371214570d05af
Author: Lucian Petrut <email address hidden>
Date: Wed Aug 28 15:58:59 2019 +0300

    Handle instances failed over while nova was down

    At the moment, the driver won't handle instances that were failed
    over while the service was down. This is due to a regression,
    the method doing this isn't called properly.

    Closes-Bug: #1841777

    Change-Id: I4ecf916b1d3323077646a38b9491785accb3c87c
    (cherry picked from commit e8e2bedd76caaa80964571834bc069937f2004dd)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/compute-hyperv 9.0.0.0rc1

This issue was fixed in the openstack/compute-hyperv 9.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.