"qemu-img convert -O qcow2 -o backing_file" makes huge images

Bug #660366 reported by Ildar
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
QEMU
Fix Released
Wishlist
Unassigned

Bug Description

$ dd if=/dev/urandom bs=1M of=1.img count=4
4+0 records in
4+0 records out
4194304 bytes (4,2 MB) copied, 1,0413 s, 4,0 MB/s
$ qemu-img create -f qcow2 -b 1.img 2.img
Formatting '2.img', fmt=qcow2 size=4194304 backing_file='1.img' encryption=off cluster_size=0
$ qemu-img convert -O qcow2 -o backing_file=1.img 2.img 3.img
$ du -h ?.img
4,1M 1.img
144K 2.img
4,3M 3.img

The conversion result is bigger then the source!

It appears that "-o backing_file" is not applied to data (as expected). I.e. all data is put into the resulting image: both from source image and "backing" image.

Expected behavior is to put only data that is not present in backing_file.

Revision history for this message
Stefan Hajnoczi (stefanha) wrote :

It is possible to chain backing files. As a workaround you could do the following:

$ qemu-img create -f qcow2 -b 2.img 4.img # from now on don't modify 2.img, instead use 4.img
$ qemu-img create -f qcow2 -b 2.img 3.img # here is the 3.img you tried to create with qemu-convert

Images 1.img and 2.img should never be modified, they are immutable snapshots.

Images 3.img and 4.img can be modified and will contain only changes against 2.img.

Perhaps qemu-img needs a command to drop data that is duplicated in the base image. This could be a new flag to commit: qemu-img commit --dedup 3.img.

Revision history for this message
Ildar (ildar-users) wrote :

Do you confirm this as a bug?

Revision history for this message
Stefan Hajnoczi (stefanha) wrote :

Sorry I'm not a frequent Launchpad user and will leave it up to someone more familiar to decide which status to place it in.

Changed in qemu:
importance: Undecided → Wishlist
status: New → Confirmed
Revision history for this message
Stefan Hajnoczi (stefanha) wrote : Re: [Qemu-devel] [Bug 660366] Re: "qemu-img convert -O qcow2 -o backing_file" makes huge images

On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori <email address hidden> wrote:
> ** Changed in: qemu
>   Importance: Undecided => Wishlist
>
> ** Changed in: qemu
>       Status: New => Confirmed

Thanks for doing this Anthony. Can I set the status myself next time
or do we have rules on who handles bugs?

Stefan

Revision history for this message
Anthony Liguori (anthony-codemonkey) wrote :

On 10/14/2010 07:51 AM, Stefan Hajnoczi wrote:
> On Thu, Oct 14, 2010 at 1:26 PM, Anthony Liguori<email address hidden> wrote:
>
>> ** Changed in: qemu
>> Importance: Undecided => Wishlist
>>
>> ** Changed in: qemu
>> Status: New => Confirmed
>>
> Thanks for doing this Anthony. Can I set the status myself next time
> or do we have rules on who handles bugs?
>

I'm pretty sure anyone can do it. If not, I'm certainly willing to give
people extra rights on the project if they want to help with bug triage.

Regards,

Anthony Liguori

> Stefan
>

Revision history for this message
Ildar (ildar-users) wrote :

I guess the problem is solved already.
qemu-img version 1.4.0

Ildar (ildar-users)
Changed in qemu:
status: Confirmed → Fix Released
Revision history for this message
sakishrist (sakishrist) wrote :

Is this reintroduced? I am on version 2.3.0

$ dd if=/dev/urandom of=disk bs=1M count=1024

$ qemu-img convert -f raw -O qcow2 disk disk.qcow

$ qemu-img convert -f raw -O qcow2 -o backing_file=disk.qcow disk disk1.qcow

$ ls -l
total 3146388
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow

All the data is copied again.

$ qemu-img info disk1.qcow
image: disk1.qcow
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G
cluster_size: 65536
*backing file: disk.qcow*
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Qemu-img works as expected though.

$ qemu-img create -f qcow2 -o backing_file=disk1.qcow disk2.qcow

$ ls -l
total 3146584
-rw-r--r-- 1 sakis sakis 1073741824 10 авг 15,29 disk
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,30 disk.qcow
-rw-r--r-- 1 sakis sakis 1074135040 10 авг 15,31 disk1.qcow
-rw-r--r-- 1 sakis sakis 197120 10 авг 15,33 disk2.qcow

Revision history for this message
Kevin Wolf (kwolf-redhat) wrote :

This is a different case. The original report used "qemu-img create" in step 2, which results in a sparse image that refers to the backing file for all data. Your sequence has "qemu-img convert" instead, which fully populates disk.qcow. Therefore, in step 3, "qemu-img convert" leaves the full allocation intact.

Revision history for this message
sakishrist (sakishrist) wrote :

My mistake. It's different case than mine. Above sequence (original report) works fine.

But I do not really understand why the same is not achieved in my case. I use the convert instead of the create to get a full image in qcow format. From that point, the desired behaviour is to create a qcow that is based on the one created from the first convert and contain only the changes. Why would the second convert populate the whole image again?

Thanks in advance.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers