soft delete instance will be reclaimed if power on failed when do restore

Bug #1841509 reported by zhangyujun on 2019-08-27
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Undecided
zhangyujun

Bug Description

I found an instance disappeared after do restore instance, check the nova code and log, I think its a logic bug here

1. restore instance with power on failed

nova-api `restore` set `instance.task_state = task_states.RESTORING instance.deleted_at = None`
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/api.py#L2344

nova-compute `restore_instance` will call `self._power_on` if virt driver did not implement the `restore` method
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L3009

instance state will be set to None if any exceptions raise when call `self._power_on` in `reverts_task_state`
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L178

finally the instnace state will be set to {vm_state=vm_state.SOFT_DELETED, task_state=None, deleted_at=None}

2. reclaim instance

nova-compute periodic task `_reclaim_queued_deletes` running every 60s,
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L8209

it will select instance with filte `{'vm_state': vm_states.SOFT_DELETED, 'task_state': None,'host': self.host}`, the instance of step 1 will be slected
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L8216

and it will be in the return list of `_deleted_old_enough` with its `deleted_at=None`
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L8430

and then be deleted soon
https://github.com/openstack/nova/blob/4b8b4217fed897755f742afcb42f7994aea4c9a1/nova/compute/manager.py#L8229

I don't think the instance should be reclaimed with the above situation

zhangyujun (zhangyujun) on 2019-08-27
Changed in nova:
assignee: nobody → zhangyujun (zhangyujun)
description: updated
Changed in nova:
status: New → In Progress
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers