rsync timestamp bug on encfs/fuse

[Kevin Conley]

Edgy
2.6.17-10-generic i686
ALL standard edgy version:
fuse (2.5.3)
encfs (1.2.5)
rsync (2.6.8)

When using rsync to encfs it will not preserve the file timestamp. This is a problem as the "quick" check will not work for rsync and thus every file is considered "new" and copied everytime.

There is a bug filed here - it seems it is fixed in newer versions for encfs/fuse?

https://bugzilla.samba.org/show_bug.cgi?id=3752

[weez (trs-e0z9g4)]

In gutsy, I am seeing this problem when copying to an encfs partition. I assume it is somehow related to this bug report:

rsync: failed to set times on "/media/usbbackup/backup/hoopy/inc-0711102037/etc/gtk/gtkrc.sr": No such file or directory (2)

[Kevin Conley (kevinmconley)]

I finally figured this out and it turned out that you have to create the encfs
with a different option. I found it on the encfs mailing list. If i get time
i'll try to look again but in th meantime you should start there.

How did you create your encfs?

I believe picking the paranoia mode sets an options that causes this. When
creating a encfs just do the (x)pert way and it should turn out ok. I think you
can also change this using encfsctl but try with a test one to make sure rsyncs
works with it before messing with encfsctl.

--- weez <email address hidden> wrote:

> In gutsy, I am seeing this problem when copying to an encfs partition.
> I assume it is somehow related to this bug report:
>
> rsync: failed to set times on
> "/media/usbbackup/backup/hoopy/inc-0711102037/etc/gtk/gtkrc.sr": No such
> file or directory (2)
>
> --
> rsync timestamp bug on encfs/fuse
> https://bugs.launchpad.net/bugs/81080
> You received this bug notification because you are a direct subscriber
> of the bug.
>

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

[weez (trs-e0z9g4)]

Hey,

I rebooted to my 6.10 (edgy?) install and was able to run my backups without any problem. I used the default (non paranoia) settings to create the encfs partition. I also experimented extensively with the expert settings and was unable to find a combination that would not result in the above error message. I think the real test would be for me to run my backup again in gutsy and see if the problem comes back since it worked once in 6.10. I will try to do that in the next 24 hours or so.

[weez (trs-e0z9g4)]

Sorry, this took a lot more than the 24 hours I promised.

So after doing the following, I am still seeing the error:
1. created encfs volume from scratch using default options
2. created one backup using 6.10 without issue
3. ran the backup in gutsy - seeing the error

A couple of lines from my latest attempt:
rsync: failed to set times on "/media/usbbackup/backup/tonkatsu/inc-0711270159/etc/alternatives/cc": No such file or directory (2)

So I am off to run the backup using 6.10.

[weez (trs-e0z9g4)]

I got tired of rebooting to run my backups once a week and so I installed a 32bit chroot edgy (6.10) install as a workaround. The backup appears to be running now with no ill effects. Just wanted to leave a quick note here in case anyone needs/wants to do this. I specifically installed fuse into the chroot, as the bug report above seemed to indicate that is where the problem is. Although now that I think about it, a 32 bit userland that has to talk to a 64 bit kernel could be problematic... I will report back if I find any problems.

I started with the instructions here:
http://ubuntuforums.org/showthread.php?t=24575

After running debootstrap, I more or less did the following (from memory):
# copied sources.list from gutsy in a non chroot shell into chroot environment
sed -i -e 's/gutsy/edgy/g' /etc/apt/sources.list
apt-get update
apt-get install language-pack-en-base
apt-get upgrade
mount -t proc proc /proc
apt-get install encfs rsync openssh-client

[Valient Gough (vgough)]

This may be due to new time functions in FUSE not implemented in encfs.. It works on the development branch, tested on x86-64 Gutsy. Will be in next release.

[Valient Gough (vgough)]

I haven't been able to reproduce this with encfs 1.4.0 on Ubuntu Gutsy x86-64 install. Marking closed.

[scrat (sam-rankin)]

Valient Gough,

When you state that this will be in the next release, are you talking 8.04? Or will an update be released for 7.10?

[Valient Gough (vgough)]

Next release of encfs. I do not maintain the Ubuntu package, or have any control over Ubuntu releases, or even Ubuntu packages of encfs.

As the previous mail said, the problem could not be reproduced with encfs 1.4.0 release. The current release is 1.4.1.1.

[Stefan Bethge (kjyv)]

Ah, great, ubuntu still has 1.3.2 in hardy... And that means all files get the current time which still makes rsync unusable. Did anybody try to compile and use 1.4.2 ? I'll try it anyway...

[Stefan Bethge (kjyv)]

I still get the same behaviour using debian packages with encfs 1.4.1.1
I created a new specific encfs dir. Copying files with rsync makes all files have the current timestamp.
I'll try to compile encfs 1.4.2 from source.

[Stefan Bethge (kjyv)]

I get the same issue with encfs 1.4.2. Is there something else I might be missing? Flags when calling encfs?

[Valient Gough (vgough)]

You may want to try upgrading to a newer version of FUSE.

I cannot reproduce this behavior - I see exactly the same timestamps when I rsync to an encfs filesystem as I do when syncing to an ext3 filesystem.

If you are still seeing this after upgrading FUSE and encfs, then please provide the output of "encfsctl info [crypt-dir]" and details of how the results are different then a normal filesystem.

[Stefan Bethge (kjyv)]

Ok, I got it working. It is though not because of Fuse which I updated from version 2.7.2 to 2.7.3 but because I created the encfs folder with cryptkeeper before which might have used the paranoia option by default, I'm not sure.
Creating with "encfs <folder> <folder>" and then selecting standard mode works fine. Are there combinations of settings which can give this result or is this still a bug which then probably occurs under certain conditions?

Below are the outputs of encfsctl info for a folder created with cryptkeeper and the one that works that is created using standard mode

cryptkeeper:

Version 6 configuration; created by EncFS 1.4.2 (revision 20080813)
Dateisystem-Verschlüsselung: "ssl/aes", Version 2:1:0 (verwende 2:1:1)
Dateinamenkodierung: "nameio/block", Version 3:0:0 (verwende 3:0:1)
Schlüssellänge: 256 Bits
Blockgröße: 512 Bytes, enthält 8 Bytes MAC-Kopf
Jede Datei enthält 8 Bytes Vorspann mit einmaligen IV-Daten.
Dateinamenkodierung benutzt IV-Verkettungsmodus.
Dateidaten-IV ist mit Dateinamen-IV verkettet.

standard mode:

Version 6 configuration; created by EncFS 1.4.2 (revision 20080813)
Dateisystem-Verschlüsselung: "ssl/aes", Version 2:1:0 (verwende 2:1:1)
Dateinamenkodierung: "nameio/block", Version 3:0:0 (verwende 3:0:1)
Schlüssellänge: 192 Bits
Blockgröße: 1024 Bytes
Jede Datei enthält 8 Bytes Vorspann mit einmaligen IV-Daten.
Dateinamenkodierung benutzt IV-Verkettungsmodus.

[weez (trs-e0z9g4)]

just to followup on my original report:

i am no longer seeing the error messages I reported above in the latest hardy 8.04

[Philippe Coval (rzr)]

I am afraid I am still affected by this issue , on hardy :

FYI:

LANG=C encfsctl /home/$USER/local/encfs/

Version 5 configuration; created by EncFS 1.3.2 (revision 20040813)
Filesystem cipher: "ssl/aes", version 2:1:1
Filename encoding: "nameio/block", version 3:0:1
Key Size: 256 bits
Block Size: 512 bytes, including 8 byte MAC header
Each file contains 8 byte header with unique IV data.
Filenames encoded using IV chaining mode.
File data IV is chained to filename IV.

rsync version 2.6.9 protocol version 29

--
http://rzr.online.fr/q/backup

[Peter Cherriman (pjcherriman)]

I'm having the same problem on hardy.

It seems to only be a problem if External IV Chaining is turned on, which it is in paranoia mode.

When creating the encfs filesystem to use with rsync, use standard mode or in expert mode say no to Enable filename to IV header chaining? That seems to work for me.

Its listed as a encfs bug in the upstream http://code.google.com/p/encfs/issues/detail?id=21

[Micheal Ah Fat (micheala)]

I can confirm the bug has not been fixed in rsync 3.0.5 & encfs 1.4.2 (Ubuntu Jaunty)

The patch still works with rsync 3.0.5

Download source from http://www.samba.org/ftp/rsync/src/rsync-3.0.5.tar.gz

Extract to /tmp
cd /tmp
vi syscall.c
Search for do_rename. Apply patch & save
./configure
make install
cp /usr/local/bin/rysnc /usr/bin