xenapi: metadata updates causing rebuild and resize to fail

[John Garbutt]

When you update instance metadata during tasks where the VM is not available, such as resize or rebuild, its possible you get NotFound exceptions causing the instances to go into the Error state, which resets the Task state, which then causes instance updates to fail that are part of the long running operation.

This should not happen, as change_instance_metadata can only be called when the task_state=None, however there is clearly a window where the metadata request has been made, and the resize or rebuild has started in the mean time, and so the NotFound error is raised.

To avoid this, we can silently fail metadata updates where the VM cannot be found, as the latest metadata will be added back into the VM when it is being configured on the destination.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/39657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/39657
Committed: http://github.com/openstack/nova/commit/4d5c2f13e8ccbe2a2aaa7dac8fcd53d158a73a73
Submitter: Jenkins
Branch: master

commit 4d5c2f13e8ccbe2a2aaa7dac8fcd53d158a73a73
Author: John Garbutt <email address hidden>
Date: Thu Aug 1 10:35:31 2013 +0100

    xenapi: skip metadata updates when VM not found

    There is a race condition where change_instance_metdata requests still
    get to the compute nodes when the VM has a task in operation that means
    the VM is no longer present. This change ensures that we skip the
    metadata updates when this occurs.

    Currently, if this happens, the long running task, like rebuild or
    resize will fail because the VM task_state=None and is put into ERROR.
    After this change, it will silently fail, and the updated metadata will
    be written when the VM gets re-created as part of the long running
    operation.

    Fixes bug 1207238
    Change-Id: I75e1f93e34d3b3ab93a8e8104fd64224f72d7309