VMware: nova complained about snapshot not ready when creating VM from the snapshot

[Zhongcheng Lao]

Rally received 400 when creating a VM from the snapshot created earlier from an instance.
Rally scenario snapshot_server will wait till the snapshot becomes READY which is odd to receive such kind of error.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/runners/base.py", line 77, in _run_scenario_once
    method_name)(**kwargs) or scenario_output
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/nova/servers.py", line 267, in snapshot_servers
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/base.py", line 254, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/nova/utils.py", line 515, in _boot_servers_with_images
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 902, in create
    **boot_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 554, in _boot
    return_raw=return_raw, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 100, in _create
    _resp, body = self.api.client.post(url, body=body)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 490, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 465, in _cs_request
    resp, body = self._time_request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 439, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 433, in request
    raise exceptions.from_response(resp, body, url, method)
BadRequest: Image 6c450a07-2e61-4720-9560-ed1b6a5c438f is not active. (HTTP 400) (Request-ID: req-ec100533-0365-4816-84fa-298f8e5b8671)

Further investigation revealed that the image was deleted by nova immediately after its status turned READY as the cleanup codes in vCenter
driver may throw an exception which triggered the delete_image_on_error.

When Rally tries to boot from the image later, nova complained about the missing images and threw 400 back to the client.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/216086

[Jay Pipes (jaypipes)]

This bug report is not describing an error in Nova at all. Furthermore, then proposed patch does nothing other than ignore (with a log warning) exceptions that occur during cleanup of the temporary VM used during snapshotting operations by the vCenter driver.

Marking as Invalid (and -2'ing the patch) until I see some indication in the bug report what is actually wrong in *Nova* (not a Rally run).

-jay

[Zhongcheng Lao (zlao)]

I've changed the bug description to be more clearer.

I don't think it should be categorized s a Rally bug
as it's a very common use case to use the image after the image becomes ready.

The cleanup codes in vCenter drive may accidentally delete the image
in such a case.

Other drivers also catch *ALL* exceptions and simply swallow them
during the cleanup.
It makes less sense to still care about these exception
as the image/snapshot is ready.
https://github.com/openstack/nova/blob/master/nova/virt/hyperv/snapshotops.py#L130

[Maciej Szankin (mszankin)]

Due to lack of activity removing current assignee.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/216086
Reason: This review is > 6 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.