Duplicity doesn't save ACLs/xattrs

[Woland]

Have an ext3 filesystem mounted with -o acl,user_xattr and directory on it with some files having acls and extended attributes.

duplicity --no-encryption dir file://bk
duplicity --no-encryption file://bk res

Now files in res have no ACLs nor xattrs.

duplicity 0.6.08b (from distro and from sources)

Python 2.5.5

Debian testing GNU/Linux

aiur:/mnt/test# duplicity -v9 --no-encryption alfa/ file://bk
Using archive dir: /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e
Using backup name: 31f2abd7613ded96f788766f0cbee63e
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.cloudfilesbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Main action: inc
================================================================================
duplicity 0.6.08b (March 11, 2010)
Args: /usr/bin/duplicity -v9 --no-encryption alfa/ file://bk
Linux aiur 2.6.32.7-aiur #3 SMP Wed Feb 10 20:35:13 YEKT 2010 i686
/usr/bin/python 2.5.5 (r255:77872, Feb 1 2010, 19:53:42)
[GCC 4.4.3]
================================================================================
Using temporary directory /tmp/duplicity-a5WVEO-tempdir
Registering (mkstemp) temporary file /tmp/duplicity-a5WVEO-tempdir/mkstemp-AMb2kK-1
Temp has 249462784 available, backup will use approx 34078720.
Synchronizing remote metadata to local cache...
Deleting local /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-full-signatures.20100408T115330Z.sigtar.gz (not authoritative at backend).
Deleting local /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-full.20100408T115330Z.manifest (not authoritative at backend).
0 files exist on backend
0 files exist in cache
Extracting backup chains from list of files: []
Last full backup date: none
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e

Found 0 secondary backup chains.
No backup chains with active signatures found
No orphaned or incomplete backup sets found.
No signatures found, switching to full backup.
Using temporary directory /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-AWuhUl-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-AWuhUl-tempdir/mktemp-P4y5eN-1
Using temporary directory /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-ZR3L5J-tempdir
Registering (mktemp) temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-ZR3L5J-tempdir/mktemp-OP8eXA-1
AsyncScheduler: instantiating at concurrency 0
Registering (mktemp) temporary file /tmp/duplicity-a5WVEO-tempdir/mktemp-ArJkgN-2
Selecting alfa
Comparing () and None
Getting delta of (() alfa dir) and None
A .
Selecting alfa/file
Comparing ('file',) and None
Getting delta of (('file',) alfa/file reg) and None
A file
Selecting alfa/test
Comparing ('test',) and None
Getting delta of (('test',) alfa/test reg) and None
A test
Selecting alfa/
Comparing ('\xed\xf4',) and None
Getting delta of (('\xed\xf4',) alfa/reg) and None
A
Selecting alfa/
Comparing ('\xf2\xe5\xf1\xf2',) and None
Getting delta of (('\xf2\xe5\xf1\xf2',) alfa/ reg) and None
A
Removing still remembered temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-AWuhUl-tempdir/mktemp-P4y5eN-1
Cleanup of temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-AWuhUl-tempdir/mktemp-P4y5eN-1 failed
Removing still remembered temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-ZR3L5J-tempdir/mktemp-OP8eXA-1
Cleanup of temporary file /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-ZR3L5J-tempdir/mktemp-OP8eXA-1 failed
AsyncScheduler: running task synchronously (asynchronicity disabled)
Writing bk/duplicity-full.20100408T120817Z.vol1.difftar.gz
Deleting /tmp/duplicity-a5WVEO-tempdir/mktemp-ArJkgN-2
Forgetting temporary file /tmp/duplicity-a5WVEO-tempdir/mktemp-ArJkgN-2
AsyncScheduler: task completed successfully
Processed volume 1
Writing bk/duplicity-full-signatures.20100408T120817Z.sigtar.gz
Writing bk/duplicity-full.20100408T120817Z.manifest
3 files exist on backend
2 files exist in cache
Extracting backup chains from list of files: ['duplicity-full.20100408T120817Z.manifest', 'duplicity-full.20100408T120817Z.vol1.difftar.gz', 'duplicity-full-signatures.20100408T120817Z.sigtar.gz']
File duplicity-full.20100408T120817Z.manifest is not part of a known set; creating new set
File duplicity-full.20100408T120817Z.vol1.difftar.gz is part of known set
File duplicity-full-signatures.20100408T120817Z.sigtar.gz is not part of a known set; creating new set
Ignoring file (rejected by backup set) 'duplicity-full-signatures.20100408T120817Z.sigtar.gz'
Found backup chain [Thu Apr 8 18:08:17 2010]-[Thu Apr 8 18:08:17 2010]
--------------[ Backup Statistics ]--------------
StartTime 1270728497.40 (Thu Apr 8 18:08:17 2010)
EndTime 1270728497.42 (Thu Apr 8 18:08:17 2010)
ElapsedTime 0.01 (0.01 seconds)
SourceFiles 5
SourceFileSize 1024 (1.00 KB)
NewFiles 5
NewFileSize 1024 (1.00 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 5
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 220 (220 bytes)
Errors 0
-------------------------------------------------

Removing still remembered temporary file /tmp/duplicity-a5WVEO-tempdir/mkstemp-AMb2kK-1

[kobuki (nls424)]

I'd also like to see ACL/EA support implemented in duplicity. I'm using ACLs and EAs extensively on a few servers and I'd really like to use duplicity because its simple and effective approach and the avility to back up to Amazon S3 backend. This is the only thing keeping me away from it, I can't afford losing the extra attributes.

[Chris (9e9-cqlic-0zx)]

Yes, please. I've been using duplicity for a few months and have been pleased with how well it has worked as a no-nonsense way of backing up my home server to S3. I've recently enabled ACLs and EA support on the server (Linux w/ ext3) in order to better support Apple OS X clients via netatalk. Though I wouldn't actually lose any data, without ACL/EA support life could be painful for the OS X clients following a restore.

[Stephane Chazelas (stephane-chazelas)]

Some tar implementations like star or bsdtar/libarchive support extensions to store ACLs and extended attributes and there seems to be a python wrapper for libarchive: http://code.google.com/p/python-libarchive/

Couldn't that be used by duplicity? Those tar archives would still be usable by say GNU tar which would just warn that it doesn't understand those extensions.

$ touch a b c d
$ setfattr -n user.test a
$ setfacl -m u:nobody:r b
$ chattr +a c
$ bsdtar cf - . | tar tf -
./
tar: Ignoring unknown extended header keyword `SCHILY.fflags'
./c
tar: Ignoring unknown extended header keyword `SCHILY.acl.access'
./b
tar: Ignoring unknown extended header keyword `LIBARCHIVE.xattr.user.test'
./a
./d

[edso (ed.so)]

libarchive would introduce another platform specific binary dependency. i'd rather go for a pure python based solution.

..ede/duply.net

[Kenneth Loafman (kenneth-loafman)]

Agreed. We use tar for storage, but never call it directly, so we would
have to gather and store the xattrs/acls ourselves anyway in a format that
duplicity would understand, so no reason to change formats at this time.

On Wed, Apr 23, 2014 at 12:29 PM, edso <email address hidden> wrote:

> libarchive would introduce another platform specific binary dependency.
> i'd rather go for a pure python based solution.
>
> ..ede/duply.net
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/558385
>
> Title:
> Duplicity doesn't save ACLs/xattrs
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> Confirmed
>
> Bug description:
> Have an ext3 filesystem mounted with -o acl,user_xattr and directory
> on it with some files having acls and extended attributes.
>
> duplicity --no-encryption dir file://bk
> duplicity --no-encryption file://bk res
>
> Now files in res have no ACLs nor xattrs.
>
> duplicity 0.6.08b (from distro and from sources)
>
> Python 2.5.5
>
> Debian testing GNU/Linux
>
> aiur:/mnt/test# duplicity -v9 --no-encryption alfa/ file://bk
> Using archive dir:
> /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e
> Using backup name: 31f2abd7613ded96f788766f0cbee63e
> Import of duplicity.backends.tahoebackend Succeeded
> Import of duplicity.backends.rsyncbackend Succeeded
> Import of duplicity.backends.imapbackend Succeeded
> Import of duplicity.backends.ftpbackend Succeeded
> Import of duplicity.backends.localbackend Succeeded
> Import of duplicity.backends.webdavbackend Succeeded
> Import of duplicity.backends.hsibackend Succeeded
> Import of duplicity.backends.cloudfilesbackend Succeeded
> Import of duplicity.backends.sshbackend Succeeded
> Import of duplicity.backends.giobackend Succeeded
> Import of duplicity.backends.botobackend Succeeded
> Main action: inc
>
> ================================================================================
> duplicity 0.6.08b (March 11, 2010)
> Args: /usr/bin/duplicity -v9 --no-encryption alfa/ file://bk
> Linux aiur 2.6.32.7-aiur #3 SMP Wed Feb 10 20:35:13 YEKT 2010 i686
> /usr/bin/python 2.5.5 (r255:77872, Feb 1 2010, 19:53:42)
> [GCC 4.4.3]
>
> ================================================================================
> Using temporary directory /tmp/duplicity-a5WVEO-tempdir
> Registering (mkstemp) temporary file
> /tmp/duplicity-a5WVEO-tempdir/mkstemp-AMb2kK-1
> Temp has 249462784 available, backup will use approx 34078720.
> Synchronizing remote metadata to local cache...
> Deleting local
> /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-full-signatures.20100408T115330Z.sigtar.gz
> (not authoritative at backend).
> Deleting local
> /root/.cache/duplicity/31f2abd7613ded96f788766f0cbee63e/duplicity-full.20100408T115330Z.manifest
> (not authoritative at backend).
> 0 files exist on backend
> 0 files exist in cache
> Extracting backup chains from list of files: []
> Last full backup date: none
> Collection Status
> -----------------
> Connecting with backend: LocalBackend
> Archive dir: /root/.cache/...

[Peter Parzer (peter-parzer)]

GNU tar 1.27.1 as included in Ubuntu 14.04 does support saving xattr/acls. So why not just using it.

[Tommy Nguyen (remyabel-deactivatedaccount)]

Preserving those attributes would be useful for SELinux systems. As far I as I know, the python tarfile library doesn't support xattrs so the solution could involve extending _proc_pax or switching to another library.