snapshot_volume_backed races, could result in data corruption

Bug #1619606 reported by Matthew Booth
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Low
Unassigned

Bug Description

snapshot_volume_backed() in compute.API does not set a task_state during execution. However, in essence it does:

if vm_state == ACTIVE:
  quiesce()
snapshot()
if vm_state == ACTIVE:
  unquiesce()

There is no exclusion here, though, which means a user could do:

quiesce()
                   quiesce()
snapshot()
                   snapshot()

unquiesce() --snapshot() now running after unquiesce -> corruption
                   unquiesce()

or:

suspend()
snapshot()
  NO QUIESCE (we're suspended)
  snapshot()
                   resume()
  --snapshot() now running after resume -> corruption

Same goes for stop/start.

Note that snapshot_volume_backed() is a separate top-level entry point from snapshot(). snapshot() does not suffer from this problem, because it atomically sets the task state to IMAGE_SNAPSHOT_PENDING when running, which prevents the user from performing a concurrent operation on the instance. I suggest that snapshot_volume_backed() should do the same.

tags: added: libvirt snapshot volumes
Revision history for this message
Chris Martin (cm876n) wrote :

I believe this is an actual bug. I tested this on my debstack on in an ubuntu environment running on vmware 14.04 and when I ran the command to snapshot the instance booted from volume from two different command prompts at almost the same time. When I did this I had multiple 'cirros-rootfs' now booted on my sidebar yet when I delete the instance and it's associated images only one of the 'cirros-rootfs' volumes would be deleted.

Sean Dague (sdague)
Changed in nova:
status: New → Confirmed
importance: Undecided → Low
Jianle He (hejianle1989)
Changed in nova:
assignee: nobody → Jianle He (hejianle1989)
Eric Fried (efried)
Changed in nova:
assignee: Jianle He (hejianle1989) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.