Cannot delete vm instance if send duplicate delete requests

Bug #1308342 reported by zhangjialong
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Rajesh Tailor

Bug Description

I deployed openstack with icehouse rc1 and booted 100 vms on my nodes. After my testing, i tried to delete my vms at the same time. Then i fount all of my vms` status change to deleting but cannot be deleted. I checked my openstack, the rabbitmq-server crashed . Then i restart rabbitmq-server and my openstack nova services, sended the delete requests again and again, the vms still cannot be deleted. While , in havana, the vms can be deleted if received duplicate delete requests .
I think icehouse should handle duplicate delete requests like havana .

zhangjialong (zhangjl)
Changed in nova:
assignee: nobody → zhangjialong (zhangjl)
Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/87868

Changed in nova:
status: New → In Progress
Revision history for this message
Takahiro Shida (shida) wrote :

Duplicated delete request fixed with a purpose following report.
https://bugs.launchpad.net/nova/+bug/1248563

I think that the ignoring the duplicate delete is raise such as this problem. So, we should discuss more about this problem.

Revision history for this message
Tushar Patil (tpatil) wrote :

IMO, there are 2 ways to fix this problem.
1. nova-api, remove the check to allow instance deletion when the task_state is DELETING.
2. nova-compute, add a new periodic task to delete the instances whose task_state is deleting.

Revision history for this message
zhangjialong (zhangjl) wrote :

@Tushar Patil

Aggree with you. Your suggestion is better than mine.

Changed in nova:
assignee: zhangjialong (zhangjl) → nobody
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by zhangjl (<email address hidden>) on branch: master
Review: https://review.openstack.org/87868
Reason: According to Tushar Patil`s suggestion, I think we should fix this problem by periodic task

Sean Dague (sdague)
Changed in nova:
status: In Progress → Confirmed
importance: Undecided → High
Changed in nova:
assignee: nobody → Rajesh Tailor (rajesh-tailor)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/121800

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/121800
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=222d44532c65ddf3f26532ced217890628352536
Submitter: Jenkins
Branch: master

commit 222d44532c65ddf3f26532ced217890628352536
Author: Rajesh Tailor <email address hidden>
Date: Mon Sep 15 07:20:14 2014 -0700

    Allow force-delete irrespective of VM task_state

    Cannot delete vm instance if duplicate delete requests are sent.

    When user sends request to delete an instance, its task_state gets
    changed to 'deleting' state. When an instance task_state is already in
    'deleting' state and if at that moment the rabbitmq-server get crashed
    by some reasons, then the instance task_state remains in 'deleting'
    state and user won't be able to delete the instance forever.

    At this moment, there is only one way to delete the instances, whose
    task_state is in 'deleting' state, by restarting the nova-compute
    services where these instances are running.

    To avoid restarting the nova-compute service manually, modified the
    force-delete api to allow instance deletion irrespective of
    instance task_state.

    Added new module to use delete_types as constants and replaced all
    delete_type string occurances with new constants.

    Closes-Bug: #1308342

    Change-Id: I4d0e47662a80109ef9622d85455587d487e47c01

Changed in nova:
status: In Progress → Fix Committed
Matt Riedemann (mriedem)
tags: added: icehouse-backport-potential juno-backport-potential
removed: havana icehose
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/juno)

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/131948

Thierry Carrez (ttx)
Changed in nova:
milestone: none → kilo-1
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/163798

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/juno)

Change abandoned by Matt Riedemann (<email address hidden>) on branch: stable/juno
Review: https://review.openstack.org/131948
Reason: This broke cells RPC API on master (kilo) so we had to revert this in kilo and then fix the bug, so no point in backporting this to juno.

Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-1 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.