Hyper-V: planned vms are not cleaned up

[Lucian Petrut]

We create a planned vm during live migration when having passthrough disks attached in order to properly configure the resources of the 'new' instance.

The issue is that if the migration fails, this planned vm is not cleaned up.

Although planned vms are destroyed at a second attempt to migrate the instance, this issue had an impact on the Hyper-V CI as planned vms persisted among CI runs and vms having the same name failed to spawn, as there were file handles kept open by the VMMS service, preventing the instance path from being cleaned up.

Trace:
http://paste.openstack.org/show/536149/

[Sean Dague (sdague)]

Need an update from the hyperv team on whether this is a real bug that can be addressed

[Lucian Petrut (petrutlucian94)]

I *am* part of the Hyper-V team, trying to understand why this bug was marked as "Incomplete".

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/478572

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/478816

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/478943

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/478572
Committed: https://git.openstack.org/cgit/openstack/os-win/commit/?id=6772315353df1d986f8181e83eecbc346a8b068a
Submitter: Jenkins
Branch: master

commit 6772315353df1d986f8181e83eecbc346a8b068a
Author: Claudiu Belu <email address hidden>
Date: Wed Jun 28 20:13:41 2017 +0300

    vmutils: Planned VMs separation

    A Planned VM and a Realized VM cannot exist at the time on the
    same host, The 2 types must be treated separately.

    vm_exists will only check if the Realized VM exists.
    planned_vm_exists will only check if the Planned VM exists.
    destroy_planned_vm will only destroy the planned VM, if it exists.

    Related-Bug: #1604078

    Change-Id: I91636a82b057f566eab9887c422911163668f556

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/478816
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=9a52134eb391dc328b5de44019d45367ab49440d
Submitter: Jenkins
Branch: master

commit 9a52134eb391dc328b5de44019d45367ab49440d
Author: Claudiu Belu <email address hidden>
Date: Thu Jun 29 12:19:51 2017 +0300

    cluster: Do not destroy clustered VMs on live migration rollback

    If a VM is clustered, it should not be destroyed on rollback.

    Change-Id: Iaa07517cb60069ff2a52f60ffa661777118eb28a
    Related-Bug: #1604078

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/497798

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/498349
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=37977e19f69fd362e11089e6fdb6d792f51cb5ce
Submitter: Jenkins
Branch: master

commit 37977e19f69fd362e11089e6fdb6d792f51cb5ce
Author: Claudiu Belu <email address hidden>
Date: Fri Aug 25 12:39:21 2017 +0300

    hyperv: Cleans up live migration Planned VM

    If an instance having iSCSI volumes attached is being
    live-migrated, a Planned VM is created at the destination.
    If the live-migration fails, the Planned VM is not cleaned
    up at the destination.

    Depends-On: I91636a82b057f566eab9887c422911163668f556

    Change-Id: If62941eb44ff1a5bbf5df01f5cfd19d9008d98bb
    Closes-Bug: #1604078

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/497798
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=ab6feea1033bfa703358b0b9fe5e83e98354c507
Submitter: Jenkins
Branch: stable/pike

commit ab6feea1033bfa703358b0b9fe5e83e98354c507
Author: Claudiu Belu <email address hidden>
Date: Fri Aug 25 12:39:21 2017 +0300

    hyperv: Cleans up live migration Planned VM

    If an instance having iSCSI volumes attached is being
    live-migrated, a Planned VM is created at the destination.
    If the live-migration fails, the Planned VM is not cleaned
    up at the destination.

    Depends-On: I91636a82b057f566eab9887c422911163668f556

    Change-Id: If62941eb44ff1a5bbf5df01f5cfd19d9008d98bb
    Closes-Bug: #1604078

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/478943
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=70296431908208375677ae428a147f312b60e1ad
Submitter: Zuul
Branch: master

commit 70296431908208375677ae428a147f312b60e1ad
Author: Claudiu Belu <email address hidden>
Date: Thu Jun 29 17:00:31 2017 +0300

    hyperv: Cleans up live migration Planned VM

    If an instance having iSCSI volumes attached is being
    live-migrated, a Planned VM is created at the destination.
    If the live-migration fails, the Planned VM is not cleaned
    up at the destination.

    This patch will ensure that rollback_live_migration_at_destination
    is being called on live-migration failure for Hyper-V compute nodes,
    which will clean up the Planned VM.

    Depends-On: I91636a82b057f566eab9887c422911163668f556

    Change-Id: I4d61325793ed559dede408813ebda5aed2b0f110
    Closes-Bug: #1604078

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/602698

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/602698
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=22c404d0f02ee80d6b1dc8d9248b2ce25c2bea3a
Submitter: Zuul
Branch: stable/rocky

commit 22c404d0f02ee80d6b1dc8d9248b2ce25c2bea3a
Author: Claudiu Belu <email address hidden>
Date: Thu Jun 29 17:00:31 2017 +0300

    hyperv: Cleans up live migration Planned VM

    If an instance having iSCSI volumes attached is being
    live-migrated, a Planned VM is created at the destination.
    If the live-migration fails, the Planned VM is not cleaned
    up at the destination.

    This patch will ensure that rollback_live_migration_at_destination
    is being called on live-migration failure for Hyper-V compute nodes,
    which will clean up the Planned VM.

    Depends-On: I91636a82b057f566eab9887c422911163668f556

    Change-Id: I4d61325793ed559dede408813ebda5aed2b0f110
    Closes-Bug: #1604078
    (cherry picked from commit 70296431908208375677ae428a147f312b60e1ad)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 18.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 19.0.0.0rc1 release candidate.