ftp remote backup broken with ncftp 3.2.3
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Invalid
|
Low
|
edso |
Bug Description
I'm using duplicity 6.0.5, python 2.6.4, with ncftpput 3.2.3 on arch linux (i686), using ftp to back up to a dlink 624 router with a usb-connected drive
duplicity was not creating incremental backups ever - it always started over with a new full backup.
i'm using the duply wrapper, but the same problem exists using duplicity from the command line.
inspecting the server i found that the backup files were being placed on the ftp root directory, and not the sub directory as specified. This might not have been a big problem, except that ncftpls was looking in the correct subdirectory on the server, and finding nothing.
inspecting the calls to ncftpput, i noticed that they had the form
- ncftpput -f configfile -F -t 30 -o useCLNT=
runing a similar command directly generated errors, and using -d turned up an inability to change directory on the server.
reading the ncftpput manual suggested that the -C option isn't supported with the -f option
i changed duplicity/
- ncftpput -f configfile -F -t 30 -o useCLNT=
with the result that the backup files landed in the intended directory, and incremental backups are now working.
difference on the files
*** ftpbackend.py.old 2009-08-29 00:47:43.000000000 +1000
--- ftpbackend.py 2009-11-21 22:02:25.000000000 +1100
***************
*** 1,3 ****
--- 1,4 ----
+ # -*- coding: utf-8 -*-
# -*- Mode:Python; indent-
#
# Copyright 2002 Ben Escoto <email address hidden>
***************
*** 89,97 ****
if not remote_filename:
! remote_path = os.path.
! commandline = "ncftpput %s -m -V -C '%s' '%s'" % \
! (self.flags, source_path.name, remote_path)
def get(self, remote_filename, local_path):
--- 90,98 ----
if not remote_filename:
! remote_path = os.path.
! commandline = "ncftpput %s -m -V '%s' '%s'" % \
! (self.flags, remote_path, source_path.name)
def get(self, remote_filename, local_path):
output of duplicity --verbosity 9
$ FTP_PASSWORD="" duplicity --verbosity 9 --encrypt-
Using archive dir: /home/bill/
Using backup name: bc97b9a36bc269d
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
Import of duplicity.
NcFTP version is 3.2.3
Using temporary directory /tmp/duplicity-
Registering (mkstemp) temporary file /tmp/duplicity-
Main action: inc
=======
duplicity 0.6.05 (August 28, 2009)
Args: /usr/bin/duplicity --verbosity 9 --encrypt-
Linux wakakusa 2.6.31-ARCH #1 SMP PREEMPT Fri Oct 23 11:12:58 CEST 2009 i686 Genuine Intel(R) CPU T2500 @ 2.00GHz
/usr/bin/python 2.6.4 (r264:75706, Oct 27 2009, 06:16:59)
[GCC 4.4.1]
=======
Registering (mkstemp) temporary file /tmp/duplicity-
Temp has 79474208768 available, backup will use approx 34078720.
Reading results of 'ncftpls -f /tmp/duplicity-
Synchronizing remote metadata to local cache...
Deleting local /home/bill/
Reading results of 'ncftpls -f /tmp/duplicity-
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: FTPBackend
Archive dir: /home/bill/
Found 0 backup chains without signatures.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
No signatures found, switching to full backup.
Using temporary directory /home/bill/
Registering (mktemp) temporary file /home/bill/
Using temporary directory /home/bill/
Registering (mktemp) temporary file /home/bill/
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-
Selecting /home/bill/sys
Comparing () and None
Getting delta of (() /home/bill/sys dir) and None
A .
Selecting /home/bill/
Comparing ('.Notes',) and None
Getting delta of (('.Notes',) /home/bill/
A .Notes
Selecting /home/bill/
Comparing ('.Notes~',) and None
Getting delta of (('.Notes~',) /home/bill/
A .Notes~
Selecting /home/bill/
Comparing ('.Yaws-
Getting delta of (('.Yaws-
A .Yaws-notes.txt~
Selecting /home/bill/
Comparing ('.arch-
Getting delta of (('.arch-
A .arch-linux-
Selecting /home/bill/
Comparing ('.etc-
Getting delta of (('.etc-
A .etc-bash.
Selecting /home/bill/
Comparing ('.package-
Getting delta of (('.package-
A .package-download~
Selecting /home/bill/
Comparing ('.wiki-
Getting delta of (('.wiki-
A .wiki-notes.txt~
Selecting /home/bill/
Comparing ('.yaws~',) and None
Getting delta of (('.yaws~',) /home/bill/
A .yaws~
Selecting /home/bill/
Comparing ('65-libmtp.
Getting delta of (('65-libmtp.
A 65-libmtp.rules
Selecting /home/bill/
Comparing ('Notes-old',) and None
Getting delta of (('Notes-old',) /home/bill/
A Notes-old
Selecting /home/bill/
Comparing ('cfg-test',) and None
Getting delta of (('cfg-test',) /home/bill/
A cfg-test
Selecting /home/bill/
Comparing ('etc-bash.
Getting delta of (('etc-
A etc-bash.
Selecting /home/bill/
Comparing ('etc_network_
Getting delta of (('etc_
A etc_network_
Selecting /home/bill/
Comparing ('installed-
Getting delta of (('installed-
A installed-
Selecting /home/bill/
Comparing ('mysql_
Getting delta of (('mysql_
A mysql_index_
Selecting /home/bill/
Comparing ('package-
Getting delta of (('package-
A package-download
Selecting /home/bill/
Comparing ('rubbish',) and None
Getting delta of (('rubbish',) /home/bill/
A rubbish
Selecting /home/bill/
Comparing ('testtest',) and None
Getting delta of (('testtest',) /home/bill/
A testtest
Selecting /home/bill/
Comparing ('weekly-update',) and None
Getting delta of (('weekly-update',) /home/bill/
A weekly-update
Selecting /home/bill/
Comparing ('wiki-notes.txt',) and None
Getting delta of (('wiki-
A wiki-notes.txt
Removing still remembered temporary file /home/bill/
Cleanup of temporary file /home/bill/
Removing still remembered temporary file /home/bill/
Cleanup of temporary file /home/bill/
AsyncScheduler: running task synchronously (asynchronicity disabled)
Running 'ncftpput -f /tmp/duplicity-
Deleting /tmp/duplicity-
Forgetting temporary file /tmp/duplicity-
AsyncScheduler: task completed successfully
Processed volume 1
Running 'ncftpput -f /tmp/duplicity-
Running 'ncftpput -f /tmp/duplicity-
Reading results of 'ncftpls -f /tmp/duplicity-
0 files exist on backend
2 files exist in cache
Extracting backup chains from list of files: []
Deleting /home/bill/
--------------[ Backup Statistics ]--------------
StartTime 1258798163.87 (Sat Nov 21 21:09:23 2009)
EndTime 1258798164.05 (Sat Nov 21 21:09:24 2009)
ElapsedTime 0.18 (0.18 seconds)
SourceFiles 21
SourceFileSize 988577 (965 KB)
NewFiles 21
NewFileSize 988577 (965 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 21
RawDeltaSize 984481 (961 KB)
TotalDestinatio
Errors 0
-------
Removing still remembered temporary file /tmp/duplicity-
Removing still remembered temporary file /tmp/duplicity-
I still see duplicity using
>Running 'ncftpput -f /tmp/duplicity- 03XnIP- tempdir/ mkstemp- TXLVQl- 1 -F -t 30 -o useCLNT= 0,useHELP_ SITE=0 -m -V -C '/home/bill/.cache bc97b9a36bc269d 3bf7129f0f58a45 d1/duplicity- full.20091121T1 00905Z. manifest. gpg' 'kincha/ backups/ duplicity- 100905Z. manifest. gpg'' (attempt #1)
>/duplicity/
>full.20091121T
in your example output
>inspecting the calls to ncftpput, i noticed that they had the form 0,useHELP_ SITE=0 -m -V -C 'pathtofilefort ransfer' 'pathtofileonse rver'
>- ncftpput -f configfile -F -t 30 -o useCLNT=
>runing a similar command directly generated errors, and using -d turned up an inability to change directory on the server.
can you provide the call and the output? Add -d for verbose ftp commands.
>reading the ncftpput manual suggested that the -C option isn't supported with the -f option
can't find any such statement. Could you point me?
As I see the rationale not to use -f together with -C, (it doubles the hostname) I tried to implement your suggested changes to put & get. Unfortunately ncftpput -f syntax is
ncftpput -f login.cfg [options] remote-directory local-files
which means that all files to be uploaded have to have their final name in the temp folder. That is and will not be the case. To circumvent local copying/renaming these files get renamed during the put/get operation using the -C switch.
Having said that. Please provide further information on the error. For now I'll flag this entry as invalid.
ede/duply.net