block migration of qcow2 VMs copies all empty space

[Andrew Bogott]

I'm running openstack 2012.1 'icehouse' which, ultimately, calls down into qemu-system-x86 2.0.0+dfsg-2ubuntu1.10.

I primed the process by copying all necessary base images onto the destination host. Nonetheless, post-migration instances are much larger than the original image; the copy duplicated all the empty space that ought to have remained copy-on-write.

[Stefan Hajnoczi (stefanha)]

Did you paste the wrong Red Hat bugzilla link?
https://bugzilla.redhat.com/show_bug.cgi?id=1058173

I don't see how this BZ is related to expanded disk image files after migration.

[Andrew Bogott (andrewbogott)]

You're right, clearly that was a c/p fail. I can't for the life of me find the bug that I was trying to refer to :(

[Stefan Hajnoczi (stefanha)]

I found a relevant Fedora bug here:
https://bugzilla.redhat.com/show_bug.cgi?id=817700

[Ansgar Hegerfeld (theraser)]

There is a patch available: https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg03742.html (and those which are mentioned in the responses)

[ADMiNZ (nikolay-zadoya)]

This patch does not help us. When migrating, the same drives become thick ...

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]

[Anthony A. (y-ubuntf-r)]

On the source :

# qemu-img info VM.img
image: VM.img
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 3.4G

On the target:

# qemu-img info VM.img
image: VM.img
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 20G

I also tried with qcow2 format but it's the same bug.

All kvm migration drop sparse disk image !

How can we resolve this ?
Does a workaround exist ?

https://bugzilla.redhat.com/show_bug.cgi?id=1219541
https://bugzilla.redhat.com/show_bug.cgi?id=1248996
https://bugzilla.redhat.com/show_bug.cgi?id=1090093
https://bugzilla.redhat.com/show_bug.cgi?id=817700

[Anthony A. (y-ubuntf-r)]

I forgot to inform my test envrionement :

Ubuntu Xenial
qemu-kvm version 2.5
libvirt-bin version 1.3.1

[Thomas Huth (th-huth)]

Can you please reproduce bugs with the latest version of QEMU, i.e. currently v2.11 ? Old versions like 2.5 are not supported by the upstream QEMU project anymore, and bugs from such old versions might have been fixed already. (Otherwise, if you're only interested in the Ubuntu binary, please open a bug against the Ubuntu QEMU package instead)

[Thomas Huth (th-huth)]

Well, if you had a closer look at the BZs that you've quoted, especially BZ 1219541, you should have seen that this problem apparently has been solved in upstream QEMU version 2.9 or 2.10. So it's really just about the backlevel version 2.5 that you're using. I'm re-assigning this ticket to the Ubuntu package, maybe they can fix it for you.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Anthony A. (y-ubuntf-r)]

I tried with qemu 2.10 and libvirt 3.6.0: not bug, it works.

I tried with raw and qcow2 format, both works.

I used this repository: https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/pike-staging

But it's a staging version, i didn't find a stable qemu/libvirt repository for Ubuntu Xenial with those versions

[Anthony A. (y-ubuntf-r)]

Maybe i can use this repository :

http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/xenial-updates

[Matt John (im1crazyassmofo)]

Just want to throw a comment on here too. We are also effected by this. We are in the process of switching to KVM and lose thin/sparse disks on live migration. Unfortunately we need this to work.

We are using:
* Ubuntu Xenial
* qemu-kvm version 2.5
* libvirt-bin version 1.3.1
* Using local storage

I've tried qcow2 and raw formats with files and even a thin LVM pool. We have had the same outcome every time.

virsh migrate --live --persistent --undefinesource --copy-storage-all (and --copy-storage-inc) fails to keep the sparse file.

I've had no luck using rsync or tar to pre-copy the disks either.

Building libvirt (v3.10.0) and qemu (v12.11) from source solved the issue but is not practical for us to do this in our environment. Using the repository above that Anthony mentioned is a temporary fix in my opinion, it's using libvirt v3.6.0 and qemu v12.10.1 and also solves the migration bug. Ubuntu should release a patch for Xenial. It appears Bionic will include the working versions.

[THittesdorf (hiroofcanton)]

Please backport qemu 2.10+ and libvirt 3.6+. Having issues with migration of thin provisioned disks without shared storage. Tried qcow/raw/lvm thin all with the same results. Can confirm that the versions from https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/pike-staging do fix the issues at least with qcow disks.

[Christian Ehrhardt  (paelzer)]

The new versions are made available via the cloud-archive already on a regular base.
We backport isolated fixed that have a low regression risk for other users, but not sull versions per the SRU Policy (https://wiki.ubuntu.com/StableReleaseUpdates).

Since it seems it was identified that the version in Bionic is good, but the new feature is huge and unlikely to be backported I'll set the tasks up that way.

I consider it unlikely, but if there is a minor set of commits assocuated that might be backportable please speak up and set the Xenial task back to new when providing the commits needed.