Hyper-V: destroying an instance can fail, having in-use files

[Lucian Petrut]

In some cases, destroying an instance can fail, having the instance log files still in-use.

This can happen in the following scenario:
1. The vm is powered on
2. The vm is requested to be destroyed shortly after (less than a few seconds). This driver will now attempt to stop the serial console workers, but none exist yet
3. The instance power on event callback is run, spawning the serial console workers
4. The instance is powered off as the destroy operation continues. The serial console workers are not explicitly stopped now. The instance serial port named pipe should be stopped now, which will cause the serial workers to stop themselves shortly after, but before this happens, the driver attempts to delete the instance files, while the console logs are still in use, which will fail.

Trace: http://paste.openstack.org/show/490196/

The easiest way to avoid this issue would be to simply retry deleting the instance files if this fails, also ensuring again that the console workers are stopped. Note that this is quite unlikely to happen in a real life deployment.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/291825
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=1c16530da82571cca666d30942b11d558a92162d
Submitter: Jenkins
Branch: master

commit 1c16530da82571cca666d30942b11d558a92162d
Author: Lucian Petrut <email address hidden>
Date: Fri Mar 11 19:11:15 2016 +0200

    Avoid starting serial console workers while deleting vm files

    If an instance destroy request is received really quick after it was
    powered on, this can fail, having the console log files in use.

    This can happen if the instance power on event is received after the
    destroy operation has started, which will spawn serial console workers
    after the destroy operation tries to clean them up.

    We can avoid this situation by simply ensuring that while we delete
    the instance log files, the serial console workers are stopped and
    no other workers will be spawned.

    Change-Id: I3759b557d735cde4391f94819a32994dfb184702
    Closes-Bug: #1556189

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/320576

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/320576
Committed: https://git.openstack.org/cgit/openstack/compute-hyperv/commit/?id=47c9525814e89ac92b12f93bd54d240c79a5056a
Submitter: Jenkins
Branch: stable/liberty

commit 47c9525814e89ac92b12f93bd54d240c79a5056a
Author: Lucian Petrut <email address hidden>
Date: Fri Mar 11 19:11:15 2016 +0200

    Avoid starting serial console workers while deleting vm files

    If an instance destroy request is received really quick after it was
    powered on, this can fail, having the console log files in use.

    This can happen if the instance power on event is received after the
    destroy operation has started, which will spawn serial console workers
    after the destroy operation tries to clean them up.

    We can avoid this situation by simply ensuring that while we delete
    the instance log files, the serial console workers are stopped and
    no other workers will be spawned.

    Change-Id: I3759b557d735cde4391f94819a32994dfb184702
    Closes-Bug: #1556189
    (cherry picked from commit 1c16530da82571cca666d30942b11d558a92162d)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/387831

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/387831
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=190c9855cbf0a1572c4dd662fb7231fbb7f920c1
Submitter: Jenkins
Branch: master

commit 190c9855cbf0a1572c4dd662fb7231fbb7f920c1
Author: Claudiu Belu <email address hidden>
Date: Tue Oct 18 10:16:58 2016 +0300

    hyperv: stop serial console workers while deleting vm files

    If an instance destroy request is received really quick after it
    was powered on, destroying the instance can fail, having the
    console log files in use.

    This can happen if the instance power on event is received after
    the destroy operation has started, which will spawn serial console
    workers after the destroy operation tries to clean them up.

    We can avoid this situation by simply ensuring that while we delete
    the instance log files, the serial console workers are stopped and
    no other workers will be spawned.

    Change-Id: I95002f5bde435ba163d878ce5c2575dc5ca19dcc
    Closes-Bug: #1556189

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0rc1 release candidate.