Xenserver: Failed power_off during "nova delete" leads to undeletable instance

Bug #1029132 reported by Kevin L. Mitchell
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Kevin L. Mitchell

Bug Description

The "nova delete" call attempts to power_off the instance, which puts the instance into task_state powering_off. If the power_off fails, however (like, if the instance isn't running), deletion stops and the instance becomes undeletable. This could also affect rescues.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/10302

Changed in nova:
assignee: nobody → Kevin L. Mitchell (klmitch)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/10302
Committed: http://github.com/openstack/nova/commit/6f0fecbed3a72ce052662a6bc6441d6aac6b9d1f
Submitter: Jenkins
Branch: master

commit 6f0fecbed3a72ce052662a6bc6441d6aac6b9d1f
Author: Kevin L. Mitchell <email address hidden>
Date: Wed Jul 25 17:27:57 2012 -0500

    Allow soft deletes from any state

    At some point in the past, the compute manager's delete() method had
    its @check_instance_state() decorator updated to allow deletes to be
    performed from any combination of vm_state and task_state. This same
    change was not made to the soft_delete() method, and so soft_delete()
    is only available if the task_state is clear, meaning that instances
    that are put into ERROR state with task_state set to something are
    effectively undeletable, if FLAGS.reclaim_instance_interval is set to
    some non-zero value (which is what causes the API DELETE call to use
    soft_delete() as opposed to delete()). This patch updates the
    @check_instance_state() on soft_delete() to match that on delete(),
    and fixes bug 1029130 and bug 1029133. This patch also partially
    fixes bug 1029132, but more needs to be done to completely fix that
    bug.

    Change-Id: I78878110c696280f2bc28434433814884a526264

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
Kevin L. Mitchell (klmitch) wrote :

Fix committed above only partially fixes the bug.

Changed in nova:
status: Fix Committed → New
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/10366

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/10366
Committed: http://github.com/openstack/nova/commit/27826a84cc603772dd4fb9fc7360b27e0f52b322
Submitter: Jenkins
Branch: master

commit 27826a84cc603772dd4fb9fc7360b27e0f52b322
Author: Kevin L. Mitchell <email address hidden>
Date: Thu Jul 26 13:49:09 2012 -0500

    Allow power_off when instance doesn't exist

    Certain errors that can occur while booting an instance may leave no
    VM on the Xen hypervisor. This can make that instance undeletable,
    since the first step in soft_delete() is to power_off the instance,
    and nothing catches the NotFound error thrown if the instance doesn't
    exist. The libvirt support handles the case of a non-existant
    instance by ignoring NotFound--after all, a non-existant instance is,
    by definition, powered off, right?

    Fixes bug 1029132.

    Change-Id: I5bffe3b61fe92fc32fb70cab941fa5ecd250df77

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → folsom-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: folsom-3 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.