Block migrating an ephemeral or swap disk can result in filesystem corruption when using qcow2

Bug #1547582 reported by Matthew Booth
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Undecided
Unassigned

Bug Description

The libvirt driver uses common backing files for ephemeral and swap disks. These are generated on the local compute host by running mkfs or mkswap as appropriate. The output of these files for a particular size and format is stored in the image cache on the compute host which ran it.

When all things are equal, 2 runs of mkfs or mkswap are guaranteed never to produce identical output, because at the very least they have different uuids. When you also consider the potential for different patch levels on different compute hosts, the potential for other differences is also significant.

When block migrating an ephemeral disk, the libvirt driver copies the 'overlay' qcow2 from source to dest. Assuming that some other instance on dest also has a similar ephemeral disk, the backing file will already exist on dest. However, it is guaranteed not to be the same as the disk's original backing file for the reasons above. If this works currently, it is either by luck, or because the tiny amount of metadata originally written by mkfs or mkswap is likely to have been overwritten if it has been in use for any amount of time.

The libvirt driver should not cache the output of mkfs and mkswap. The space and performance benefits are negligible, but it introduces the potential for data corruption.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/282433

Changed in nova:
assignee: nobody → Matthew Booth (mbooth-9)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/282434

Revision history for this message
Matthew Booth (mbooth-9) wrote :

Dan Berrange pointed out that while generating ext4 filesystems is fast and uses a trivial amount of space, generating ext3 is very slow and uses vastly more space. There's more discussion to be had before making this change.

Revision history for this message
Maciej Szankin (mszankin) wrote :

Last change was abandoned and there is no current progress - removing assignee.

Changed in nova:
assignee: Matthew Booth (mbooth-9) → nobody
status: In Progress → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.