instance's host isn't updated if nova-compute stops operating during live-migration

[Eli Qiao]

I do a live-migration from host1 to host2. If during live migration nova-compute service stops operating it will leave things in messy state. Even when live migration ends successfully from hypervisor perspective, nova does not update VM's host to new one. It also leaves VIFs and volume connections on source host.

nova list show that the instance is still active (actually it is because live migration has completed. However, nova loses track of a VM)

taget@liyong:~/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+----------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+----------------------+
| 1d114104-9a62-49ba-b209-6a42beff4133 | test1 | ACTIVE | - | NOSTATE | private_net=10.0.0.9 |

'nova show test1' shows that this instance is still on host1 (because service nova-compute on host1 crashed and it had no chance to update it).

After that any nova operation on this instance fails because nova tries to execute every operation on old host (after such fail nova changes instance state to error).

[Eli Qiao (taget-9)]

I think for this issue, currently there isn't any good solution sto fix this issue since the status is mainly controled by the source host,
if source host is crashed/or be killed by mistake /or lose power.

so, I am thinking if we can add some logic in periodic task to correct instance's state per compute's real state??

[lvmxh (shaohef)]

Eli,

You "nova list" the VM on destination host, did you do more actions to check anything about the migrated VM works well?

So what do you expect? Just update the instance 's state on source host?

[Pawel Koniszewski (pawel-koniszewski)]

Shaohe, this issue is not only about instance's host not being updated in database. After successful live migration it also leaves volume connection open to both, source host and destination host. Basically it needs to run post live migration operations on source host and destination host.

[Eli Qiao (taget-9)]

yeah, pawel , you are right, so this isn't a easy fix, what I am thinking about is we need a "supervisor" or "task flow" (maybe in conductor to watch the whole process of live-migration.

[Eli Qiao (taget-9)]

another blue print is working on https://blueprints.launchpad.net/nova/+spec/manager-restart-during-migration