Comment 0 for bug 1005901

Revision history for this message
nocturo (nocturo) wrote :

I've experienced issues restoring backups when /tmp is tmpfs partition. I've tried this on older (0.6.15) and latest duplicity (0.6.19), python 2.4.2 (CentOS 5 default), python 2.7.3 (built from source) and it's always using /tmp for some parts of the process while I specifically define TMPDIR which is taken in effect somewhat.

during earlier stages of restore, it doesn't use /tmp and suddenly starts using /tmp. lsof reports the file in /tmp is deleted and I can't actually see what's it's doing there. Here's debug (9) output of the restore process when this happens:

Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HUu1tG-22
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HUu1tG-22
Processed volume 4 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-yLMJml-23
Processed volume 5 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-RmEejg-24
Processed volume 6 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-8mpG2M-25
Processed volume 7 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-0l0tGX-26
Processed volume 8 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-tmyNCY-27
Processed volume 9 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28
Deleting /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28
Forgetting temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-dYo9xJ-28
Processed volume 10 of 145
Registering (mktemp) temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-n68Y0Z-29
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-BLRdqu-19
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-TYZNJs-13
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-z7pTc1-11
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-n68Y0Z-29
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-YndPPI-15
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-fkl21c-5
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-HEGXbH-7
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-oTmsi3-3
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-Ky2n3b-18
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-UgfduZ-10
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-YH3N28-17
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-ou7sde-16
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-AAQhlE-14
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-nvni05-6
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mkstemp-H5WCdY-1
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-eUCf1V-12
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-OlulTA-8
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-O9HkIO-9
Removing still remembered temporary file /home/.duplytmp/duplicity-yVLVCU-tempdir/mktemp-sZj6Lb-4
Traceback (most recent call last):
  File "/usr/local/bin/duplicity", line 1391, in <module>
    with_tempdir(main)
  File "/usr/local/bin/duplicity", line 1384, in with_tempdir
    fn()
  File "/usr/local/bin/duplicity", line 1318, in main
    restore(col_stats)
  File "/usr/local/bin/duplicity", line 623, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 522, in Write_ROPaths
    for ropath in rop_iter:
  File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 495, in integrate_patch_iters
    final_ropath = patch_seq2ropath( normalize_ps( patch_seq ) )
  File "/usr/local/lib/python2.7/site-packages/duplicity/patchdir.py", line 475, in patch_seq2ropath
    misc.copyfileobj( current_file, tempfp )
  File "/usr/local/lib/python2.7/site-packages/duplicity/misc.py", line 170, in copyfileobj
    outfp.write(buf)
IOError: [Errno 28] No space left on device

I first thought this was something in python itself defaulting to /tmp, but it also happened on newest python too so I'm fairly sure duplicity fails here. I'm only starting learning python so can't debug more/write patches so I depend on you guys.

lsof output looks like this:
duplicity 13755 root 3r REG 0,21 104835547 25612 /duplybackup/self/duplicity-full.20120509T115438Z.vol63.difftar.gz (x:/backup/node/vm)
duplicity 13755 root 4r REG 8,1 60151379 20433546 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-ZwHDoQ-3
duplicity 13755 root 5r REG 8,1 78092697 20433547 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-otjk9B-4
duplicity 13755 root 6r REG 8,1 51332868 20433548 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-Li5Lea-5
duplicity 13755 root 7r REG 8,1 56067864 20433549 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-b1RRyo-6
duplicity 13755 root 8r REG 8,1 80891540 20433550 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-6F880l-7
duplicity 13755 root 9w REG 8,1 104833635 20433551 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-WAoXTy-8
duplicity 13755 root 10w REG 8,1 71777882 20433552 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-ipj0ni-9
duplicity 13755 root 11r REG 8,1 48588498 20433553 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-SDiDK_-10
duplicity 13755 root 12r REG 8,1 38507998 20433554 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-RDECXu-11
duplicity 13755 root 13r REG 8,1 41653195 20433555 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-isRCM4-12
duplicity 13755 root 14r REG 8,1 45080364 20433556 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-UlTLTG-13
duplicity 13755 root 15w REG 8,1 95725089 20433557 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-GrIAel-14
duplicity 13755 root 16r REG 8,1 87851979 20433558 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zTQNfz-15
duplicity 13755 root 17w REG 8,1 104835141 20433559 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-mvtG4c-16
duplicity 13755 root 18w REG 8,1 67628505 20433560 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-i0XhUm-17
duplicity 13755 root 19w REG 8,1 63160613 20433561 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-hnjypn-18
duplicity 13755 root 20w REG 8,1 29449321 20433562 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zNErBv-19
duplicity 13755 root 21w REG 7,1 4665376768 48959 /tmp/tmpflxctko (deleted)
duplicity 13755 root 22w REG 8,1 46923776 20433545 /home/.duplytmp/duplicity-f4aRJ9-tempdir/mktemp-zHKiCL-29