qemu-img convert intermittently corrupts output images
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Cinder |
Won't Fix
|
High
|
Tony Breeds | ||
OpenStack Compute (nova) |
Won't Fix
|
High
|
Tony Breeds | ||
QEMU |
Fix Released
|
Undecided
|
Tony Breeds | ||
qemu (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Trusty |
Fix Released
|
High
|
Unassigned | ||
Utopic |
Fix Released
|
High
|
Unassigned | ||
Vivid |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
=======
Impact: occasional image corruption (any format on local filesystem)
Test case: see the qemu-img command below
Regression potential: this cherrypicks a patch from upstream to a not-insignificantly older qemu source tree. While the cherrypick seems sane, it's possible that there are subtle interactions with the other delta. I'd really like for a full qa-regression-test qemu testcase to be run against this package.
=======
-- Found in releases qemu-2.0.0, qemu-2.0.2, qemu-2.1.0. Tested on Ubuntu 14.04 using Ext4 filesystems.
The command
qemu-img convert -O raw inputimage.qcow2 outputimage.raw
intermittently creates corrupted output images, when the input image is not yet fully synchronized to disk. While the issue has actually been discovered in operation of of OpenStack nova, it can be reproduced "easily" on command line using
cat $SRC_PATH > $TMP_PATH && $QEMU_IMG_PATH convert -O raw $TMP_PATH $DST_PATH && cksum $DST_PATH
on filesystems exposing this behavior. (The difficult part of this exercise is to prepare a filesystem to reliably trigger this race. On my test machine some filesystems are affected while other aren't, and unfortunately I haven't found the relevant difference between them, yet. Possible it's timing issues completely out of userspace control ...)
The root cause, however, is the same as in
http://
and it can be solved the same way as suggested in
http://
In qemu, file block/raw-posix.c use the FIEMAP_FLAG_SYNC, i.e change
f.fm.fm_flags = 0;
to
f.fm.fm_flags = FIEMAP_FLAG_SYNC;
As discussed in the thread mentioned above, retrieving a page cache coherent map of file extents is possible only after fsync on that file.
See also
https:/
In that bug report filed against nova, fsync had been suggested to be performed by the framework invoking qemu-img. However, as the choice of fiemap -- implying this otherwise unneeded fsync of a temporary file -- is not made by the caller but by qemu-img, I agree with the nova bug reviewer's objection to put it into nova. The fsync should instead be triggered by qemu-img utilizing the FIEMAP_FLAG_SYNC, specifically intended for that purpose.
Changed in qemu (Ubuntu): | |
status: | New → Triaged |
importance: | Undecided → High |
Changed in qemu (Ubuntu Trusty): | |
status: | New → Triaged |
importance: | Undecided → High |
description: | updated |
description: | updated |
tags: |
added: verification-done-trusty removed: verification-needed |
tags: | added: verification-needed-utopic |
Changed in mos: | |
status: | New → Triaged |
importance: | Undecided → Critical |
assignee: | nobody → MOS Linux (mos-linux) |
milestone: | none → 6.0 |
tags: |
added: verification-done-utopic removed: verification-needed-utopic |
Changed in cinder: | |
importance: | Undecided → High |
Changed in cinder: | |
status: | In Progress → Triaged |
assignee: | Tony Breeds (o-tony) → nobody |
Changed in cinder: | |
assignee: | nobody → Tony Breeds (o-tony) |
Changed in nova: | |
milestone: | none → kilo-2 |
Changed in nova: | |
milestone: | kilo-2 → kilo-3 |
Changed in nova: | |
milestone: | kilo-3 → none |
Is there a minimum version of qemu that would be required to use the FIEMAP_FLAG_SYNC flag?