"qemu-img convert -O qcow2 -o backing_file" makes huge images

[Ildar]

$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M 1.img
144K 2.img
4,3M 3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

[Stefan Hajnoczi (stefanha)]

It is possible to chain backing files. As a workaround you could do the following:

$ qemu-img create -f qcow2 -b 2.img 4.img # from now on don't modify 2.img, instead use 4.img
$ qemu-img create -f qcow2 -b 2.img 3.img # here is the 3.img you tried to create with qemu-convert

Images 1.img and 2.img should never be modified, they are immutable snapshots.

Images 3.img and 4.img can be modified and will contain only changes against 2.img.

Perhaps qemu-img needs a command to drop data that is duplicated in the base image. This could be a new flag to commit: qemu-img commit --dedup 3.img.

[Ildar (ildar-users)]

Do you confirm this as a bug?

[Stefan Hajnoczi (stefanha)]

Sorry I'm not a frequent Launchpad user and will leave it up to someone more familiar to decide which status to place it in.

[Stefan Hajnoczi (stefanha)]

On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori <email address hidden> wrote:
> ** Changed in: qemu
>   Importance: Undecided => Wishlist
>
> ** Changed in: qemu
>       Status: New => Confirmed

Thanks for doing this Anthony. Can I set the status myself next time
or do we have rules on who handles bugs?

Stefan

[Anthony Liguori (anthony-codemonkey)]

On 10/14/2010 07:51 AM, Stefan Hajnoczi wrote:
> On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori<email address hidden> wrote:
>
>> ** Changed in: qemu
>> Importance: Undecided => Wishlist
>>
>> ** Changed in: qemu
>> Status: New => Confirmed
>>
> Thanks for doing this Anthony. Can I set the status myself next time
> or do we have rules on who handles bugs?
>

I'm pretty sure anyone can do it. If not, I'm certainly willing to give
people extra rights on the project if they want to help with bug triage.

Regards,

Anthony Liguori

> Stefan
>

[Ildar (ildar-users)]

I guess the problem is solved already.
qemu-img version 1.4.0

[sakishrist (sakishrist)]

Is this reintroduced? I am on version 2.3.0

$ dd if=/dev/urandom of=disk bs=1M count=1024

$ qemu-img convert -f raw -O qcow2 disk disk.qcow

$ qemu-img convert -f raw -O qcow2 -o backing_file=disk.qcow disk disk1.qcow

$ ls -l
total 3146388
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow

All the data is copied again.

$ qemu-img info disk1.qcow
image: disk1.qcow
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
*backing file: disk.qcow*
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Qemu-img works as expected though.

$ qemu-img create -f qcow2 -o backing_file=disk1.qcow disk2.qcow

$ ls -l
total 3146584
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow
-rw-r--r-- 1 sakis sakis 197120 10 авг 15,33 disk2.qcow

[Kevin Wolf (kwolf-redhat)]

This is a different case. The original report used "qemu-img create" in step 2, which results in a sparse image that refers to the backing file for all data. Your sequence has "qemu-img convert" instead, which fully populates disk.qcow. Therefore, in step 3, "qemu-img convert" leaves the full allocation intact.

[sakishrist (sakishrist)]

My mistake. It's different case than mine. Above sequence (original report) works fine.

But I do not really understand why the same is not achieved in my case. I use the convert instead of the create to get a full image in qcow format. From that point, the desired behaviour is to create a qcow that is based on the one created from the first convert and contain only the changes. Why would the second convert populate the whole image again?

Thanks in advance.