delete vm after shelve vm

[Boxiang Zhu]

Description
===========
I delete vm after I have done shelve action for vm.

Notes: I set reclaim_instance_interval = 604800 in nova.conf

Steps to reproduce
==================
It's very important to set reclaim_instance_interval = 604800 in nova.conf

Then do steps as followed:

    $ nova boot --flavor <flavor-id> --image <image-id> --nic net-id=<net-id> <name>
    $ nova shelve <vm-id>
    $ nova delete <vm-id>
    $ nova force-delete <vm-id>
    ERROR (CommandError): No server with a name or ID of '856407c6-2334-4470-a1fe-d57e33535b99' exists.

Expected result
===============
The vm can be force deleted correctly.

Actual result
=============
The vm can not be found.

Find the vm from the db as followed:

    mysql> use nova_cell1;
    mysql> select vm_state,uuid,deleted from instances where uuid="856407c6-2334-4470-a1fe-d57e33535b99"\G;
               vm_state: soft-delete
                 uuid: 856407c6-2334-4470-a1fe-d57e33535b99
                deleted: 4

We will find that the 'deleted' field is not 0.

Environment
===========
Deploy with master of devstack.

Logs & Configs
==============
No error from log.

nova.conf config with reclaim_instance_interval = 604800

[Balazs Gibizer (balazs-gibizer)]

Why do you expect that after you delete the VM, the VM still can be force-deleted? I think the nova delete command already deleted the VM so there is nothing to be force-deleted.

Marking this as Invalid. Feel free to set it back to New if you disagree.

[Boxiang Zhu (bxzhu-5355)]

Hi gibizer, I have set reclaim_instance_interval = 604800 in nova.conf. So when we call nova `delete` command to delete vm, in fact it means to soft delete the vm. And we can call `force-delete` to delete vm at last.

[Balazs Gibizer (balazs-gibizer)]

I assume you deployment does not use positive shelved_offload_time value in the config. So the shelve command puts is in SHELVED_OFFLOADED state immediately. That means instance.host will be None and therefore nova-api will do a local delete [1]. The soft delete codepath would to a call down to the compute to stop the VM but without instance.host pointing to any host that is not done. This is why the instance got fully deleted if it is in SHELVED_OFFLOADED state before the delete.

Base on the code structure I don't think nova ever considered soft deleting a SHELVED_OFFLOADED VMs. I'm not sure we can change this behavior as a bugfix as users might already depend on the current behavior. So this needs to be handled as a new feature. [2]

[1] https://github.com/openstack/nova/blob/edaaa97d9911b849f3b5731746274b44a08ce14c/nova/compute/api.py#L2440
[2] https://docs.openstack.org/nova/latest/contributor/process.html#how-do-i-get-my-code-merged

[Boxiang Zhu (bxzhu-5355)]

yes, I deploy the openstack cluster with devstack by default config. And the default value of shelved_offload_time is 0. Thank you, gibizer.

BTW, do we consider that when vm is in SHELVED_OFFLOADED state, nova refuses to soft-delete vm?

[Balazs Gibizer (balazs-gibizer)]

I think rejecting soft delete in SHELVED_OFFLOADED state would be even stranger behavior. I think the way forward is to propose a specification where it clarified and where the team can agree what would be the expected behavior in case of soft-delete of a SHELVED_OFFLOADED instance.