Ironic Nova driver makes two calls to delete a node

Bug #1436568 reported by Josh Gachnang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
Medium
Lucas Alvares Gomes
OpenStack Compute (nova)
Fix Released
Medium
Lucas Alvares Gomes

Bug Description

When deleting an instance in Nova, it sets the provision state to DELETED and then when that completes (node is in CLEANING, CLEANFAIL, or NOSTATE/AVAILABLE), it makes another call to remove the instance UUID. The instance UUID should be cleared out when Ironic clears out node.instance_info, and Nova should delete the instance as soon as the node is one of the states above.

Tags: ironic
Josh Gachnang (joshnang)
Changed in ironic:
assignee: nobody → Josh Gachnang (joshnang)
Josh Gachnang (joshnang)
Changed in nova:
assignee: nobody → Josh Gachnang (joshnang)
tags: added: ironic
Changed in nova:
importance: Undecided → Low
Dmitry Tantsur (divius)
Changed in ironic:
status: New → Triaged
importance: Undecided → Medium
Changed in nova:
status: New → Confirmed
Revision history for this message
Lucas Alvares Gomes (lucasagomes) wrote :

Hi Josh,

I think the problem here is that Nova relies on finding a node with that instance_uuid when waiting the provision state after marking the node to be deleted [1], if that bit just handled NodeNotFound and gracefully exit that loop (assuming Ironic had cleaned the instance_uuid) it wouldn't be a problem.

I'm going to put up a patch to do it.

[1] https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L809

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/204995

Changed in nova:
assignee: Josh Gachnang (joshnang) → Lucas Alvares Gomes (lucasagomes)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/205033

Changed in ironic:
assignee: Josh Gachnang (joshnang) → Lucas Alvares Gomes (lucasagomes)
status: Triaged → In Progress
Changed in nova:
importance: Low → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/204995
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5eddd50537f1666021f71b2df7f49d58400ec31f
Submitter: Jenkins
Branch: master

commit 5eddd50537f1666021f71b2df7f49d58400ec31f
Author: Lucas Alvares Gomes <email address hidden>
Date: Thu Jul 23 10:54:12 2015 +0100

    Ironic: Better handle InstanceNotFound on destroy()

    After telling Ironic to delete the instance the destroy() code in the
    Ironic nova driver keeps waiting for the provision state of that node
    to change. If the instance have been already cleaned up in Ironic the
    nova code should just ignore it and continue. This also skip trying to
    update the node to remove the instance association if the instance was
    already removed in Ironic.

    Missing unittests to the _unprovision() method were added as part of
    this patch since it touches it.

    Closes-Bug: #1436568
    Change-Id: I0b1b710056d48c8b7bb2b46fdaba192922926420

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/205033
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=2894bcd84960e4133c8bd8552878e7e2c7efb240
Submitter: Jenkins
Branch: master

commit 2894bcd84960e4133c8bd8552878e7e2c7efb240
Author: Lucas Alvares Gomes <email address hidden>
Date: Thu Jul 23 13:15:47 2015 +0100

    Clean up instance_uuid as part of the node's tear down

    Prior to this patch Ironic would expect Nova to clean up the instance
    association, but this assumption does not play well with the standalone
    version of Ironic or cleaning.

    This patch keeps a backward compatibility layer with old versions of Nova
    that will try to clean up the instance_uuid field even if it's already
    cleaned.

    Depends-On: I0b1b710056d48c8b7bb2b46fdaba192922926420
    Closes-Bug: #1436568
    Change-Id: I58bc848ab3a60d3e9ab16d00ce96a603df1c244a

Changed in ironic:
status: In Progress → Fix Committed
Changed in ironic:
milestone: none → 4.0.0
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: none → liberty-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: liberty-3 → 12.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.