With 70% free space duplicity dies with [Errno 28] No space left on device:

[Wojciech Adam Koszek]

I have the Synology DS214play with Atom CPU and 1GB of RAM. I have RAID mirror with 2x1TB disks, and 2 USB volumes attached: 320GB and 1TB disk. In this report I attempt to test-drive duplicity on 1TB volume by running:

root@wkoszek_nas:/mnt/volumeUSB2# time duplicity --no-encryption /home/wkoszek file://`pwd`/backup-duplicity.20151005 > duplicity.report.txt
[Errno 28] No space left on device: '/mnt/volumeUSB2/backup-duplicity.20151005/duplicity-full.20151006T085704Z.vol13107.difftar.gz'

real 1724m51.360s
user 1449m55.449s
sys 118m55.726s

Report file:

Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Tue Oct 6 01:55:12 2015
RESTART: The first volume failed to upload before termination.
         Restart is impossible...starting backup from beginning.
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.

Mount points:

/dev/root 2451064 1594404 754260 68% /
/tmp 358092 180 357912 1% /tmp
/run 358092 2544 355548 1% /run
/dev/shm 358092 0 358092 0% /dev/shm
/volume1/homes 956675772 922960672 33612700 97% /volume1/@appstore/debian-chroot/var/chroottarget/home
/volumeUSB1/usbshare 312494912 153154016 159340896 50% /volume1/@appstore/debian-chroot/var/chroottarget/mnt/volumeUSB1
/volumeUSB2/usbshare 976251072 335877424 640373648 35% /volume1/@appstore/debian-chroot/var/chroottarget/mnt/volumeUSB2
/dev 355860 4 355856 1% /volume1/@appstore/debian-chroot/var/chroottarget/dev

I run duplicity from chroot'ed Debian environment, with duplicity installed via apt-get. Version:

root@wkoszek_nas:/mnt/volumeUSB2# duplicity --version
duplicity 0.6.24

Dir entry to the backup directory:

drwxrwxrwx 2 1024 users 2097152 Oct 7 06:41 backup-duplicity.20151005

---------------- this paragraph is wrong and written by mistake ----------

I couldn't figure out what's wrong, but I feel this is a problem:

root@wkoszek_nas:/mnt/volumeUSB2# du -ch backup-duplicity.20151005/
320G backup-duplicity.20151005/
320G total

root@wkoszek_nas:/mnt/volumeUSB2# ls -1 backup-duplicity.20151005/ | wc -l
13106

And some random websites state that VFAT (which I believe indicates FAT32) has a low limit of files in a directory:

http://superuser.com/questions/446282/max-files-per-directory-on-ntfs-vol-vs-fat32

------------------------------------------------------------------------

This question remains the same:

Is there are a good reason why duplicity is storing all files in 1 directory?

[Wojciech Adam Koszek (wkoszek)]

Last paragraph of my bug report is incorrect. I confused 13106 with something close to 131072, but 1 zero is missing in the 1st number.

[Kenneth Loafman (kenneth-loafman)]

All files in one directory is an old design limitation. It can be worked
around by using something like --volsize=500, which will set the volume
size to 500MB instead of the default of 25MB, thus reducing the number of
volumes by 20x. You can go higher depending on the limits of the target
filesystem, I think 2000MB for VFAT.

On Thu, Oct 8, 2015 at 12:27 AM, Wojciech A. Koszek <email address hidden>
wrote:

> Last paragraph of my bug report is incorrect. I confused 13106 with
> something close to 131072, but 1 zero is missing in the 1st number.
>
> ** Description changed:
>
> I have the Synology DS214play with Atom CPU and 1GB of RAM. I have RAID
> mirror with 2x1TB disks, and 2 USB volumes attached: 320GB and 1TB disk.
> In this report I attempt to test-drive duplicity on 1TB volume by
> running:
>
> root@wkoszek_nas:/mnt/volumeUSB2# time duplicity --no-encryption
> /home/wkoszek file://`pwd`/backup-duplicity.20151005 > duplicity.report.txt
> [Errno 28] No space left on device:
> '/mnt/volumeUSB2/backup-duplicity.20151005/duplicity-full.20151006T085704Z.vol13107.difftar.gz'
>
> real 1724m51.360s
> user 1449m55.449s
> sys 118m55.726s
>
> Report file:
>
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup left a partial set, restarting.
> Last full backup date: Tue Oct 6 01:55:12 2015
> RESTART: The first volume failed to upload before termination.
> - Restart is impossible...starting backup from beginning.
> + Restart is impossible...starting backup from beginning.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: none
> No signatures found, switching to full backup.
>
> Mount points:
>
> /dev/root 2451064 1594404 754260 68% /
> /tmp 358092 180 357912 1% /tmp
> /run 358092 2544 355548 1% /run
> /dev/shm 358092 0 358092 0% /dev/shm
> /volume1/homes 956675772 922960672 33612700 97%
> /volume1/@appstore/debian-chroot/var/chroottarget/home
> /volumeUSB1/usbshare 312494912 153154016 159340896 50%
> /volume1/@appstore/debian-chroot/var/chroottarget/mnt/volumeUSB1
> /volumeUSB2/usbshare 976251072 335877424 640373648 35%
> /volume1/@appstore/debian-chroot/var/chroottarget/mnt/volumeUSB2
> /dev 355860 4 355856 1%
> /volume1/@appstore/debian-chroot/var/chroottarget/dev
>
> I run duplicity from chroot'ed Debian environment, with duplicity
> installed via apt-get. Version:
>
> root@wkoszek_nas:/mnt/volumeUSB2# duplicity --version
> duplicity 0.6.24
>
> + ---------------- this paragraph is wrong and written by mistake
> + ----------
> +
> I couldn't figure out what's wrong, but I feel this is a problem:
>
> root@wkoszek_nas:/mnt/volumeUSB2# du -ch backup-duplicity.20151005/
> 320G backup-duplicity.20151005/
> 320G total
>
> root@wkoszek_nas:/mnt/volumeUSB2# ls -1 backup-duplicity.20151005/ | wc
> -l
> 13106
>
> And some random websites state that VFAT (which I believe indicates
> FAT32) has a low limit of files in a directory:
>
> http:/...