sbackup's tar very slow due to erroneous backup size calculation

[Thibault Godouet]

I run Ubuntu 11.04 with sbackup 0.11.4-0ubuntu2.

Yesterday I ran a backup with sbackup. It took it almost 18 hours to backup 30GB from a SATA disk to an eSATA disk. This kind of backup would normally take way way less: probably around one hour.

I believe the problem was that sbackup somehow thought there was pretty much nothing to backup (150ko):

011-10-15 15:35:34,303 - INFO: L'espace libre maximum requis est '0 Mo 150 Ko 512'.

while in fact the tar file is 30GB:

Total bytes written: 31355320320 (30GiB, 444KiB/s)

Because of this it ran tar as:

/bin/tar --gzip -cS --directory=/ --ignore-failed-read --blocking-factor 20 --totals --files-from=/tmp/sbackup/includes.list --exclude-from=/tmp/sbackup/excludes.list --checkpoint=1 --checkpoint-action=exec=/usr/share/sbackup/sbackup-progress --listed-incremental=/tmp/sbackup/files.snar

Please note the --checkpoint=1. This is completely inappropriate in this case, and it result in tar forking to run backup-progress every 10KB. This slows down tar dramatically. I proved this by replacing backup-progress by /bin/true, which made the end of the backup much faster.

Looking at ar_backend/tar.py, it appears this --checkpoint value was generated based on the size of the backup which was wrong. It would make sense to understand further why that size was wrong (let me know if I can help), but I think the code should be more defensive here anyway, to make things more robust.

I thus propose the attached patch to avoid dramatic performance problem if the checkpoint calculation goes wrong. The value of 100 is arbitrary: I chose it as it means tar would report its progress every 1MB, and it seems to me nobody will care about more frequent progress updates on nowadays machines. But of course you may choose to tweak this value a bit, as long as it can never get close to 1.

Thanks,
Thibault.

ps: this is slightly unrelated but is there some documentation on all the config parameters available? for more advanced users it would be very useful, as people like me find convenient to edit the config files in vi rather than having to load a GUI. A simple webpage and/or file in /usr/share/doc would do, even if a man page would be nicer.

[Thibault Godouet (tglaunchpad)]

See sbackup's logs attached.

Please note that sbackup doesn't end backuping 30GB anymore when I run it again now that the previous backup completed successfully.