Restore fails with “too many open files”

[Martin Ueding]

I use duplicity with Backblaze B2, have an incremental backup every hour and a full backup every 90 days. I recently tried to restore a file to the state that it had 14 days ago. Eventually it failed saying “too many open files”.

Apparently the last full backup was 2020-03-08 15:00:05Z. So that means that _a lot_ of incremental backups needed to be resolved. These are the last lines of output:

Get: duplicity-inc.20200414T190005Z.to.20200414T200005Z.vol1.difftar.gpg -> temp/duplicity-a7n2sb9u-tempdir/mktemp-619jt1ub-252
Verarbeitung des lokalen Manifest b'/home/mu/.cache/duplicity/17532b308f659397a23dae3345234484/duplicity-inc.20200414T200005Z.to.20200414T210005Z.manifest' (7297)
1 Volumen in Manifest gefunden
Get: duplicity-inc.20200414T200005Z.to.20200414T210005Z.vol1.difftar.gpg -> temp/duplicity-a7n2sb9u-tempdir/mktemp-z7996evy-253
Verarbeitung des lokalen Manifest b'/home/mu/.cache/duplicity/17532b308f659397a23dae3345234484/duplicity-inc.20200414T210005Z.to.20200414T220005Z.manifest' (5983)
1 Volumen in Manifest gefunden
Get: duplicity-inc.20200414T210005Z.to.20200414T220005Z.vol1.difftar.gpg -> temp/duplicity-a7n2sb9u-tempdir/mktemp-alqkws80-254
Traceback (innermost last):
  File "/usr/bin/duplicity", line 100, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 86, in with_tempdir
    fn()
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_main.py", line 1538, in main
    do_backup(action)
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_main.py", line 1618, in do_backup
    restore(col_stats)
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_main.py", line 723, in restore
    if not patchdir.Write_ROPaths(globals.local_path,
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 578, in Write_ROPaths
    for ropath in rop_iter:
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 541, in integrate_patch_iters
    for patch_seq in collated:
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 407, in yield_tuples
    setrorps(overflow, elems)
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 396, in setrorps
    elems[i] = next(iter_list[i])
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 126, in difftar2path_iter
    tarinfo_list = [next(tar_iter)]
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 355, in __next__
    self.set_tarfile()
  File "/usr/lib64/python3.8/site-packages/duplicity/patchdir.py", line 348, in set_tarfile
    self.current_fp = next(self.fileobj_iter)
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_main.py", line 759, in get_fileobj_iter
    yield restore_get_enc_fileobj(backup_set.backend,
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_main.py", line 804, in restore_get_enc_fileobj
    fileobj = tdp.filtered_open_with_delete(u"rb")
  File "/usr/lib64/python3.8/site-packages/duplicity/dup_temp.py", line 125, in filtered_open_with_delete
    fh = FileobjHooked(path.DupPath.filtered_open(self, mode))
  File "/usr/lib64/python3.8/site-packages/duplicity/path.py", line 802, in filtered_open
    return gpg.GPGFile(False, self, gpg_profile)
  File "/usr/lib64/python3.8/site-packages/duplicity/gpg.py", line 225, in __init__
    p1 = gnupg.run([u'--decrypt'], create_fhs=gnupg_fhs,
  File "/usr/lib64/python3.8/site-packages/duplicity/gpginterface.py", line 374, in run
    process = self._attach_fork_exec(gnupg_commands, args,
  File "/usr/lib64/python3.8/site-packages/duplicity/gpginterface.py", line 403, in _attach_fork_exec
    pipe = os.pipe()
 OSError: [Errno 24] Too many open files

I would assume that it stumbles over the 1024 file descriptor limit and it just cannot resolve a backup with that many incremental backups. Basically with the current state of the software my incremental backups are worthless because I cannot restore. Is there something that I can do except doing less frequent incremental updates and more full backups?

[Kenneth Loafman (kenneth-loafman)]

This is a known issue with duplicity. You will need to increase your systems number of open files since it's set so low. I'd suggest 16384 or higher if you have a lot of incrementals. I'd also suggest limiting the number of incrementals and doing more full backups.