VMware: nova complained about snapshot not ready when creating VM from the snapshot

Bug #1487950 reported by Zhongcheng Lao on 2015-08-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Undecided
Unassigned

Bug Description

Rally received 400 when creating a VM from the snapshot created earlier from an instance.
Rally scenario snapshot_server will wait till the snapshot becomes READY which is odd to receive such kind of error.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/runners/base.py", line 77, in _run_scenario_once
    method_name)(**kwargs) or scenario_output
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/nova/servers.py", line 267, in snapshot_servers
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/base.py", line 254, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/benchmark/scenarios/nova/utils.py", line 515, in _boot_servers_with_images
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 902, in create
    **boot_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/v1_1/servers.py", line 554, in _boot
    return_raw=return_raw, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 100, in _create
    _resp, body = self.api.client.post(url, body=body)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 490, in post
    return self._cs_request(url, 'POST', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 465, in _cs_request
    resp, body = self._time_request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 439, in _time_request
    resp, body = self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 433, in request
    raise exceptions.from_response(resp, body, url, method)
BadRequest: Image 6c450a07-2e61-4720-9560-ed1b6a5c438f is not active. (HTTP 400) (Request-ID: req-ec100533-0365-4816-84fa-298f8e5b8671)

Further investigation revealed that the image was deleted by nova immediately after its status turned READY as the cleanup codes in vCenter
driver may throw an exception which triggered the delete_image_on_error.

When Rally tries to boot from the image later, nova complained about the missing images and threw 400 back to the client.

Zhongcheng Lao (zlao) on 2015-08-24
Changed in nova:
assignee: nobody → Zhongcheng Lao (zlao)

Fix proposed to branch: master
Review: https://review.openstack.org/216086

Changed in nova:
status: New → In Progress
Changed in nova:
assignee: Zhongcheng Lao (zlao) → Gary Kotton (garyk)
Changed in nova:
assignee: Gary Kotton (garyk) → Zhongcheng Lao (zlao)
Jay Pipes (jaypipes) wrote :

This bug report is not describing an error in Nova at all. Furthermore, then proposed patch does nothing other than ignore (with a log warning) exceptions that occur during cleanup of the temporary VM used during snapshotting operations by the vCenter driver.

Marking as Invalid (and -2'ing the patch) until I see some indication in the bug report what is actually wrong in *Nova* (not a Rally run).

-jay

Changed in nova:
status: In Progress → Incomplete
Zhongcheng Lao (zlao) on 2016-06-13
description: updated
Zhongcheng Lao (zlao) wrote :

I've changed the bug description to be more clearer.

I don't think it should be categorized s a Rally bug
as it's a very common use case to use the image after the image becomes ready.

The cleanup codes in vCenter drive may accidentally delete the image
in such a case.

Other drivers also catch *ALL* exceptions and simply swallow them
during the cleanup.
It makes less sense to still care about these exception
as the image/snapshot is ready.
https://github.com/openstack/nova/blob/master/nova/virt/hyperv/snapshotops.py#L130

Maciej Szankin (mszankin) wrote :

Due to lack of activity removing current assignee.

Changed in nova:
assignee: Zhongcheng Lao (zlao) → nobody
Changed in nova:
status: Incomplete → Confirmed

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/216086
Reason: This review is > 6 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers