Duplicity cannot handle sparse files efficiently

[Aki Tuomi]

When running backup with duplicity, it cannot handle sparse files efficiently.

I have a virtual machine image on my home dir,

26G -rw-r--r-- 1 libvirt-qemu kvm 33G huhti 28 08:59 dev02.img

And duplicity is backing it up 33G, when it should only back up 26G.

Steps to reproduce

1. Use qemu-image to create a sparse file
2. Run duplicity
3. Observe duplicity back it up whole instead of very small size

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Sven Mueller (smu-u)]

It seems that the change to support this would be minimal, see below. All file writing during restore seems to be done by util.py/copyfileobj()

Unless I'm very much mistaken, the following change would do sparse files, _if_ (and only if) the target object is empty to begin with. If it isn't, this will keep blocks with old content when it shouldn't(big caveat).

             if all([ b == 0 for b in buf ]):
                 outfp.seek(len(buf), 1) # Move forward len(buf) bytes
             else:
                 bytes_written += len(buf)
                 outfp.write(buf)

as the replacement for what previously were the last two lines of the above.

[Kenneth Loafman (kenneth-loafman)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

 status confirmed
 importance medium
 done
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEExwf+p6/3mDvUSsaGH1G0ZBEHM/EFAmBmFfMACgkQH1G0ZBEH
M/FnDQgAhImWqh76T9WUH4NzzfarEEZsdLKXg6lspkK14v3nEV0J1kd4PnsgxmL0
k0c2q3vcUOuB0QTyydWb5Jzf4C6JzmIRZEkJaHNojU7oFgyzu+AFq3r56/UdVFh/
yw84Ly3vaxN25BlR+T5kHENHtzmp9sMS8KdcECwh7EfxtxluItF/bkXk8kYukhlO
WhnEkiyw/Fb6+5sQ0tl73t+V9zLOn2WrTLpSxaAsiZMel33aOBYgoPOic0v+Wpsv
B407MFxAen5Z4EKHWwpQ2HLHik6OXiir6ZVX400+6inSJflqBCobJM0Lm/NH8OKu
NacHNc7wXXHVGI9vKMliwg36OTJvCg==
=/NIB
-----END PGP SIGNATURE-----