(enhancement) backup to samba share, sshfs, ftp etc

[Dick Thomas]

It would be epically useful to be able to back to to samba shares other other file system shares now that we can back up to fat32

[Guy Van Sanden (gvs)]

This would be a very important feature for me too as backups on the local machine are only limited in use.

[Vreemde eend (pieter-cogghe)]

I think Time Machine resolves the lack of hard links with a virtual filesystem (sparse disk image?) that is mounted. I don't know if such a concept exists in Linux, but if so, it might solve the problems with remote filesystems like sshfs?

[Adam Reeve (adreeve)]

I think this would be quite useful. At the moment I just use fstab to mount a samba share but it would be nice if Back In Time could mount shares itself when needed. My samba share is on another Linux machine so hard links aren't a problem. I think perhaps support for file systems that don't support hard links should be a separate feature request to this one.

Would the best way to implement this be to temporarily mount the samba share using mount.cifs? On Ubuntu suid is set to root so regular users can mount samba shares. Is this the case in other distros or is there a better way for users to mount samba shares? Or can the .deb/.rpm make sure suid of mount.cifs is set to root?

[Adam Reeve (adreeve)]

I've added support for mounting network shares if any one is interested in trying it out. It can't be configured graphically yet though, and only supports samba shares. I'm also pretty inexperienced with this sort of thing so might this might not be the best way to implement it. The branch is at https://code.launchpad.net/~adreeve/backintime/samba_access.

[Kayle S (kslays)]

It's already working great for me with NFS on 10.04 Lucid Lynx. I edited my /etc/fstab to include my NFS mounts, and just set up Back in Time to target the mounted directory. I'd been trying to get sbackup (Simple Backup Suite/Solution) working, but I couldn't figure out how, even reading lots of guides. Now we just need Ubuntu to provide a GUI for setting up NFS so you don't have to edit /etc/fstab (and this would be nice for samba etc. too).

[cuc (cuc+)]

gvfs maybe? like nautilus uses to mount shares

[perlhead (fheinz)]

Actually, rsync already provides all the needed functionality for taking snapshots through the net via ssh without the drawbacks of sshfs (mainly, lack of hard links, due to the fact that the sftp protocol does not support them).

The problem seems to be that 1) backintime does not provide a way to specify that the backup should be done via ssh, and 2) the program expects to be able to access the snapshots vie filesystem operations for stuff it does outside of rsync.

It seems to me that the solution to this would involve abstracting all snapshot access through plugins that can route operations to the appropriate protocol (the filesystem plugin would access the files through the OS, the ssh plugin would use sftp, etc), and letting rsync do its magic for snapshot taking.

[Dan (danleweb)]

This is planned for version ... 2.0 : add a mount / umount (ou mount + rsync).

[perlhead (fheinz)]

Dan, adding mount/umount does not solve the problem for sshfs, only for SMB, NFS and the like. The problem is, sshfs works over the SFTP protocol, which lacks support for hard links. This means that the whole hard-link magic fails when you use sshfs. So screw sshfs, use a real filesystem, you say? Not so fast: ssh is a great tool, and makes it really easy to set up secure backup “in the cloud” as they say.

No problem with using mount/umount for restore operations (that works), but backintime needs to support rsync directly over ssh (no mounting needed) in order to be useful in this setup.

[Dan (danleweb)]

The problem is than to take a snapshot BIT use "mkdir", "cp" (for hardlinks) and "rsync".
I'll have to find a way to manage to use sshfs & rsync in the same process.

[perlhead (fheinz)]

Not really. You can do mkdir over ssh ('ssh host mkdir foo'), and let rsync do the hard linking. You don't need sshfs at all for taking the snapshot.

[Dan (danleweb)]

rsync "--link-dest" option don't work for BIT because it check user/group/rights too:
1. BIT make snapshots read-only
2. some file systems don't support linux user/group/rights, like NTFS
3. rsync can't always set the correct u/g/r, ex: you backup a file that belong to root as a regular user

I didn't find a way to force it (it would make my file easier).

NOTE: BIT save u/g/r in a special file.

[perlhead (fheinz)]

user-group rights should not be a problem:

1. if you run rsync as an unprivileged user, you should not be backing up other people's files
2. If you are backing up data for several users, then your rsync should be running as root... in which case your rsync will be able to manipulate user/group settings.

The real problem is, of course, when you try to use some brain-dead filesystem which doesn't support users and groups. I really think that it's better not to support those filesystems than to give up rsync over ssh.

[Christoph Buchner (bilderbuchi)]

if I followed this discussion right, some of these problems are already solved by rdiff-backup (i.e. usage of cp and mkdir, remote filesystem detection, more?).
http://www.nongnu.org/rdiff-backup/index.html
http://www.nongnu.org/rdiff-backup/features.html
[I know, it's a kind of cross-post from bug #587588, it just fits in here, too, though.

[perlhead (fheinz)]

Christoph,

actually, as I said below, rsync already has everything we need for performing the backup. rdiff-backup also does the trick, but it stores snapshots in a different way (probably more efficient storage-wise, but more computationally intensive when going back to older revisions).

Changing BackInTime to support rsync over ssh to perform the backup is less work than getting it to work with rdiff-backup.

[Dan (danleweb)]

BIT should be able to easily explore snapshots and open/view files (read only).
I don't know if this is so easy using rdiff-backup.

[matt (matthiasgies)]

For KDE, KIO compatibility should do the job, i guess.

[Oceanwatcher (oceanwatcher)]

I am running a Zentyal server in-house (Ubuntu based with a nice web-gui) and I would love to be able to back files from different computers in the house to the server. But as BIT does not support shares on a server, this is very difficult.

I think this is so important that it deserves a high priority. Also, I think BIT should be part of the standard KDE package so that all distros that use KDE has a decent backup system!

[ceg (ceg)]

Duplicity as backend uses rdiff-backup and would bring network support but also space efficient incremental diffs and encryption features http://duplicity.nongnu.org/

[Anders Wallenquist (aw)]

It seems like backintime uses hard links at the target, not between the source and the snapshots. This means that it would be easy to move the taget directory to a server using for example GIO/gvfs. GIO uses urls for bookmarks, and I believe that the url for a ssh target could be used with a rsync-command with little work. If its ssh-mounts only is ok for us.

Harlinks (same inode-number) in snabshots, another inode for the source-file.

aw@biblioteket:~/Backup/backintime$ ls -i 20110224-100001/backup/home/aw/openerp-client_6.0.1-0_all.deb
6424056 20110224-100001/backup/home/aw/openerp-client_6.0.1-0_all.deb
aw@biblioteket:~/Backup/backintime$ ls -i 20110213-000002/backup/home/aw/openerp-client_6.0.1-0_all.deb
6424056 20110213-000002/backup/home/aw/openerp-client_6.0.1-0_all.deb
aw@biblioteket:~/Backup/backintime$ ls -i /home/aw/openerp-client_6.0.1-0_all.deb
5374595 /home/aw/openerp-client_6.0.1-0_all.deb
aw@biblioteket:~/Backup/backintime$

[Trenton D. Adams (trent-trentonadams)]

My thoughts are purely from an SSH point of view, and I'm not sure how they would integrate with a general solution for network backups. I'm kind of under the impression that each network backup type would kind of need to be implemented on it's own.

One question I would have off the top of my head though; why not drop the permission saving feature completely, when doing unix based backups, if desired? Is it so that the backup is completely portable? If it is, do the permission saving thing, but still leave the file system with proper permissions in tact, perhaps?

Dan, you did mention issues with hard linking and --link-dest, regarding permissions. Doesn't --chmod solve that problem, assuming the same --chmod is used through the life-time of the network backups? And, if the --chmod is changed, maybe execute a shell call to change permission to what you want them to be?

Additionally, I think a solution for this would need to store meta-data for the remote backups locally, potentially for a variety of reasons
1. efficiency of displaying available backups
2. efficiency during the backup process, as you don't have to grab remote files to check what the last backup was, for example.

p.s.
It would be ideal for this feature to use password less ssh keys. The user can ensure that no other commands can be run on the remote side, via the authorized keys file.

[Nils Herde (hernil)]

How is this feature comming along?

For me it's really one of the only things holding BIT back and the one feature I miss! :-)

[Michal Pěnka (michal-penka)]

I'm on the same wave...Anybody has an update on SSHFS support?

[Eric JACQUOT (ejacquot)]

Hi all,

I think using sshfs to Back-In-Time is a bad and wrong idea :

First, there are some systems doesn't support FUSE ans SSHFS.
Second, Backintime seems to be an embedded rsync frontend. What about rsync over ssh ?

No time to paste any diff and don't use the trunk branch, but it will be usefull to do in this way. ;)

[Silvano (silvavlis)]

I agree with Eric JACQUOT, why not using rsync over ssh?

[Germar (germar)]

I've added a branch providing ssh access for BackinTime. Till now there is only Gnome support. Comments and Bug reports are very welcome.
https://code.launchpad.net/~germar/backintime/ssh