Comment 2 for bug 810493

2011/7/14 Vish Ishaya <email address hidden>:
> It is possible to put a qcow2 image into glance if you are using kvm.

It's debatable whether qcow2 images really are sparse :) but
nevertheless, I think Glance should support this more natively. There
is a *lot* of disk space and network traffic to be saved here.

If we take the Ubuntu 11.04 cloud image as an example:
$ ls -l ubuntu-11.04
-rw-r--r-- 2 glance nogroup 1476395008 2011-07-07 14:43 ubuntu-11.04

# Create a sparse version:
$ cp --sparse=always ubuntu-11.04 ubuntu-11.04.sparse

# a tar'ed version:
$ tar cvSf ubuntu-11.04.sparse.tar ubuntu-11.04.sparse

# Both gzip+tar:
$ tar cvzSf ubuntu-11.04.sparse.tar.gz ubuntu-11.04.sparse

$ ls -ls ubuntu-11.04*
1441824 -rw-r--r-- 2 soren soren 1476395008 2011-07-07 14:43 ubuntu-11.04
 567012 -rw-r--r-- 1 soren soren 1476395008 2011-07-15 15:14
ubuntu-11.04.sparse
 478856 -rw-r--r-- 1 soren soren 490342400 2011-07-15 15:29
ubuntu-11.04.sparse.tar
 172208 -rw-r--r-- 1 soren soren 176339294 2011-07-15 15:24
ubuntu-11.04.sparse.tar.gz

Unsurprisingly, the worst sinner is the fully allocated disk image.
It's costly to store and costly to transfer over the network.

The sparsely allocated image is quite a bit better in terms of storage
space on the Glance host, but doesn't save any network bandwidth.

The tar'ed version further reduces the allocated size by quite a bit
and also reduces the amount of data that needs to be transferred over
the network.

The tar+gzipped version, though, is the clear winner. It's only a
fraction of the size of the other images and also saves a *lot* of
bandwidth.

Just as an extra data point, I also added a gzip'ed version of the
original image:
 174448 -rw-r--r-- 1 soren soren 178628893 2011-07-15 15:34 ubuntu-11.04.gz

..but before anyone suggests just adding support for Content-Encoding:
gzip, remember that this means that Glance would have to process the
full image (in this case, a reasonably measly 1.4 GB image, but
potentially much larger images) and compress it in-flight. The
receiving end would also need to do special tricks to maintain the
image's sparseness, as gunzip doesn't care to create sparse files on
unpack. Adding tar to the mix means that only 490342400 bytes need to
get gzip'ed, and tar natively handles packing/unpacking sparse images
effectively.

--
Soren Hansen        | http://linux2go.dk/
Ubuntu Developer    | http://www.ubuntu.com/
OpenStack Developer | http://www.openstack.org/