'metadata' file not found when creating backup ("Could not restore ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"

Bug #1217959 reported by Tom Slominski
512
This bug affects 110 people
Affects Status Importance Assigned to Milestone
Déjà Dup
Fix Released
Medium
Unassigned
duplicity (Ubuntu)
Incomplete
Undecided
Unassigned

Bug Description

Linux release: Ubuntu 13.04 64bit
Kernel: 3.8.0-29-generic
Python: 2.7.4
Deja Dup: 26.0
Duplicity: 0.6.21

Hi. I've tried to do my monthly or so backup recently, but Deja Dup crashed after trying to backup a very large file (VM disk with Windows 7, 50GB or so). I tried to do another one, this time excluding all my large VM's. However, now when trying to do a backup I'm getting the following error a few seconds after inputting my encryption password:

Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found in backup

I've tried reinstalling both deja-dup and duplicity, purging duplicity, deleting deja-dup and duplicity from /home/tom/.cache, deleting the backup files on the external drive, deleting it's config as described here http://askubuntu.com/questions/53980/how-to-delete-all-the-settings-for-deja-dup. I know (well, I have some evidence supporting this) that the drive itself isn't faulty as I managed to backup my old laptop to the drive using duplicity and it worked fine (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs momentarily.

Tags: 14.04
Revision history for this message
Tom Slominski (tomslominski) wrote :
description: updated
Revision history for this message
Tom Slominski (tomslominski) wrote :
Revision history for this message
Tom Slominski (tomslominski) wrote :

Doesn't that delete basically all the config for the up to date programme that use dconf?

Revision history for this message
Tom Slominski (tomslominski) wrote :

Normal duplicity command line backups work fine, so this is probably a Deja Dup bug.

summary: - 'metadata' file not found when creating backup
+ 'metadata' file not found when creating backup ("Could not restore
+ ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
Changed in deja-dup:
status: New → Confirmed
Revision history for this message
Alistair Grant (akgrant0710) wrote :

I've seen this bug only once, after I paused the backup and re-started it. The following backup completed successfully.

Revision history for this message
Michael Terry (mterry) wrote :

Do not run the command in comment #3! As Tom notes, that will delete a lot of configuration data for all of your programs, not just Deja Dup.

This error is happening during Deja Dup's test restore. In each backup, it includes this tiny file and after each backup, it tries to restore it. This is a way to verify that the backup is restorable on at least a basic level.

Tom, from your log, it looks like your backup got interrupted/stopped? So it was incomplete, but then Deja Dup tried its verification process anyway and (obviously) had a problem restoring from the incomplete backup. Does that ring a bell?

Changed in deja-dup:
importance: Undecided → Medium
status: Confirmed → Incomplete
Revision history for this message
Tom Slominski (tomslominski) wrote :

I highly doubt that it got interrupted. The only way it could've been interrupted is hard drive issues, which is unlikely as I don't remember the drives becoming inaccessible afterwards. However, my hard drive enclosure is held together with a rubber band, which says it all really.

I'll be doing more backups as soon as possible so I'll get back to you then about whether this is still an issue.

Revision history for this message
Simon Langley (simon-ouryard) wrote :

It happens everytime I do a backup. Maybe interrupting deja-dup is a problem but from what you've said why would it persist?

Revision history for this message
Michael Terry (mterry) wrote :

Simon, I can't speak to your issue. I was looking at the logs provided by Tom, who opened this bug.

This error message might happen for several reasons, and I would need to see the log to determine why.

Revision history for this message
Tom Slominski (tomslominski) wrote :

Nope, still getting this. It happened on Ubuntu 13.10, deja-dup 27.3.1-0ubuntu1, duplicity 0.6.21-0ubuntu4.1. I started doing an initial, encrypted backup and it came up with a restore test: http://kwl.me/ss/195620.png I entered my password and it then came up with the error I described before. This time I was backing up to an external btrfs partition.

Revision history for this message
Tom Slominski (tomslominski) wrote :

Definitely still happening, while a duplicity backup works fine. I can try the nightlies next time I do a backup. Would that be helpful? What kind of logs would you need if I was to do so?

Revision history for this message
Helmut Neubauer (helmut-8) wrote :

I've the same problem to. I stopped a backup manually per GUI. Now, no backup works.
Nothing helps. I tried to delete .cache/deja-dup, I tried to initialize a new backup at a new location, but the problem still exists.

Revision history for this message
Paul Drain (pd) wrote :

Had this on a fresh Ubuntu 14.04 Daily Image too (2014-02-12):

I had configured the backup settings as normal (to local, freshly formatted EXT4 USB stick), and closed the window -- half an hour later, the error popped up on the screen.

However, if I remove my deja-dup preferences from dconf and delete the .cache/deja-dup directory and try the process on a fresh stick _and_ press "Backup Now" (rather than letting it do it), the backup goes through & works fine.

Both a selected restore (from Nautilus) and a full restore (from Deja Dup, to an empty directory on the host drive) work from both sticks without errors though.

What's the preferred way to provide debug logs to find the problem?

Revision history for this message
WoodyEckelzone (bcr497) wrote :

Also have this every time. 13.10

Revision history for this message
WoodyEckelzone (bcr497) wrote :

BTW my home folder is on SSD and
~/.cache is symlinked to an harddrive

Can that cause the problem?

Revision history for this message
Brandon P. (brandonmpace) wrote :

I replicated this on 14.04 by symlinking ~/.cache to another partition. (home on SSD)
All was fine before this.

Revision history for this message
Sébastien Lerique (wehlutyk) wrote :

I also have this bug on Debian Jessie, but note that in the settings I am excluding ~/.cache from the files to back up. If Deja Dup is testing a restore with a file from ~/.cache, it's logical that it fails. Do any of the people above also have ~/.cache excluded from the backups? If not, I'll file a new bug since this is different.

$ uname -a
Linux iona 3.13-1-amd64 #1 SMP Debian 3.13.10-1 (2014-04-15) x86_64 GNU/Linux

$ dpkg -s deja-dup
Package: deja-dup
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 5597
Maintainer: Debian GNOME Maintainers <email address hidden>
Architecture: amd64
Version: 30.0-1
Depends: libatk1.0-0 (>= 1.12.4), libc6 (>= 2.4), libcairo-gobject2 (>= 1.10.0), libcairo2 (>= 1.2.4), libgdk-pixbuf2.0-0 (>= 2.22.0), libgirepository-1.0-1 (>= 0.9.2), libglib2.0-0 (>= 2.37.3), libgtk-3-0 (>= 3.9.12), libnautilus-extension1a (>= 2.91), libnotify4 (>= 0.7.0), libpango-1.0-0 (>= 1.14.0), libpangocairo-1.0-0 (>= 1.14.0), libpeas-1.0-0 (>= 1.0.0), libsecret-1-0 (>= 0.7), dconf-gsettings-backend | gsettings-backend, duplicity (>= 0.6.23)
Recommends: gvfs-backends, python-boto (>= 0.9d), python-cloudfiles, ssh-client | openssh-client
Conffiles:
 /etc/xdg/autostart/deja-dup-monitor.desktop d0d75ac59827808e870fadd2e59b4e72
Description: [...]

Revision history for this message
Michael Terry (mterry) wrote :

~/.cache is always excluded anyway. As I mentioned in comment #7, this is a deja-dup internal sanity check on your backup. It creates a file in ~/.cache and includes it in each backup as a canary-in-the-coal-mine.

Revision history for this message
Maarten Vergauwen (maarten-vergauwen) wrote :

I tried it on a fedora 20 machine with the same result.

Similar to comment #16, my home dir is actually a symbolic link to another disk.
IMO chances are high this triggers the bug.

versions on fedora:
deja-dup-30.0-1.fc20.x86_64
3.14.4-200.fc20.x86_64

Revision history for this message
Joe (jgerm54654378955) wrote :

Like others, this bug only occurred for me once I changed my cache directory to a symbolic link to another drive.

Revision history for this message
rubo77 (rubo77) wrote :

Please someone with therights delete the comment #3! As Tom notes, that will delete a lot of configuration data for all of your programs, not just Deja Dup.

Revision history for this message
Jim Caughran (caughranjim) wrote :

Note that like others, my .cache is a link to a larger disk. Ubuntu 14.4

I wondered if an empty file would suffice, and created a file in .cache/deja-dup/. I started deja-dup from the windowskey launcher. It asked if I wanted a password on the backups, then started to create "first backup". I checked the folder, and discovered that it had removed all previous backups. In .cache, it converted by empty file to a folder with a single file in it, README:

     This folder can be safely deleted.
     1405284829

It's working on the backup now. If it fails, I'll change this.

Revision history for this message
Jim Caughran (caughranjim) wrote :

Nope. After the backup,

       Could not restore ‘/home/jim/.cache/deja-dup/metadata’: File not found in backup

The folder it created disappeared.

Revision history for this message
Bib (bybeu) wrote :

14.04 fresh install. Trying to backup ("weekly") my whole home to external (usb) dedicated NTFS drive I also meet this error stating I should "remove" my backup. Deleted all partitions and reformat the whole as ext4, retried, same error. Then I tried excluding the huge part (my pictures tree). This time it worked, so I went on, removing some folders out of exclusion list, and forcing "backup now" ... worked ok.... went on until time to go to bed when I shut the PC down (until this moment, deja-dup main tab showed "Next backup will happen tomorrow). This morning I want to go on, remove some more folders out of exclusion list.... and the error is here again :( (this time I'm not advised to remove the backup) ... I retried, keeping the exclusion list as is, and this time it worked again. I can't remember what the schedule info was this morning when I first ran deja-dup, but at the moment it is "Next backup will happen in 7 days". I go on each time removing some sub-folders out of the exclusion list, and it still seems to work.
Maybe a schedule triggering issue?

Bib (bybeu)
Changed in deja-dup:
status: Incomplete → Confirmed
tags: added: 14.04
Revision history for this message
Bib (bybeu) wrote :

I compared with a OK-running other laptop and this maybe strange diff in gsettings:

OK laptop
org.gnome.DejaDup.File relpath @ay []

KO laptop
org.gnome.DejaDup.File relpath b''

Revision history for this message
Bib (bybeu) wrote :

repost clean
Comment 26 for bug 1217959
Bib (bybeu) wrote 1 hour ago: #26

Yesterday in the evening I changed the backup target from the usb drive to a network folder though ssh. At this time the proposed/previous target was "Sauvegarde". I didn't choose the proposed remote $HOME/.../deja-dup/ target, but instead a customized [remote]/$HOME/.Backup Studio1537. Then I changed the exclusion list, just to exclude .mozilla folder (which as it's own cloud backup regards to Firefox sync), then re-enabled weekly backup and when prompted entered the password to store.
During the backup the laptop went to sleep state several times until I disabled sleep feature and he went on backing-up until this morning ~4h AM, but no time since backup start deja-dup stated an error. Only in the end displayed the error stating "Backup failed" Unable to **RESTORE** /home/marie/.cache/deja-dup/metadata: file is not in the backup.
The $HOME is on a SSD and this is no external .cache symlink or any here in (single drive in the laptop). At the moment I still have the failure popup displayed but I can't find any log.
Attached are list on the target, showing backup ran until the end (~40GB data), a list of local deja-dup folder and settings:
Have a look at time/date/size to make sure ~start time and end time.

I can add that only once in the process I had a look at "Details" expanding the list, then I closed it, feeling this was maybe involved in previous failures I experienced.

Revision history for this message
Daniel Koć (kocio) wrote :

For me the root of the problem was interrupted backup. Removing .cache/deja-dup dodd not help, but after:

dconf reset -f /org/gnome/deja-dup/

(and making references once again) the backup starts. So, probably the problem lies in some bad dconf entries after not succesful backup process.

Revision history for this message
gaolugang (gaolugang) wrote :

#29 works for me. My problem was also that I interrupted the previous backup and this problem keeps happening after that.

Revision history for this message
gregor (gregor-6) wrote :

hi,

i have her the same problem. i tried as destination local file and ssh.

environment: ubuntu 14.04 x86_64 (kernel 3.13.0-35)
deja-dup 30.0

a reset of the dconf settings + deleting .cache/deja-dup did not help

Revision history for this message
Simon Zwölfer (simon-zwoelfer) wrote :

Hi, just the same for another two environments. Like in Post #31 reset of the dconf settings + deleting .cahce/deja-dup did not help. Another fact is, that a test-restore of a file that should be included in the backup via nautilus does not work, too.

Revision history for this message
Dario Nuevo (wee-xbe) wrote :

I'm in the same boat. LTS 14.04, reset dconf and deleting .cache/deja-dup didn't help..

Also, I've observed an odd behavior: In my case, it doesn't actually backup anything. It starts a fresh backup (when I've reset everything), and after some seconds skips to "verifying" directly, without having done a backup. Paths are OK; my user home is quite big. I don't know if I'm the only one having this behavior?

If somehow deja-dup isn't able to backup anything, it also makes sense that it doesn't find the .cache-test-file in the backup of course ;-)

Important: In my case, my whole /home/[user] is a bind mount (`mount -o bind`) to another disk - could that be an issue?

** Is there any way to get the duplicity commands what deja-dup is using, or any other method to know what it's actually doing? Any way to see under the GUI? Any logs, any --verbose switch? (didn't see any..?)

Revision history for this message
Dario Nuevo (wee-xbe) wrote :

It seems deja-dup (or duplicity) has a problem with backup folders that are either symlinks and/or bind mounts.

Please see my comment #33 above regarding my situation. Here, /home/[user] is symlinked to /mnt/data/[user] and no backup is performed - it just skips it failing with the error this issue here is about. Most other people here say, they symlinked/over-mounted ~/.cache. In their case, it backups and then fails. In my case, it fails immediately.

As a test, i added /mnt/data/[user] (the source of the bind mount; thus the same files) to the backup folders. It is now backing up.. it takes some time as it's initial.. But i bet it will also fail with the same error, as ~/.cache is still on the bind mount (as /home/dn [the bind mount target] is still in the backup set)..

I think we should rename this issue to clarify.. I suggest duplicity/deja-dup are not correctly handling symlinks and/or bind mounts. Just a guess, but would make sense.. how to be sure?

Revision history for this message
Dario Nuevo (wee-xbe) wrote :

Ah, one correction to my last post (no edit function?)

I wrote
> Here, /home/[user] is symlinked to /mnt/data/[user]
It should be
> Here, /home/[user] is a bind mount (`mount -o bind`) of /mnt/data/[user]

Revision history for this message
Nathaniel Homier (mechamechanism) wrote :

What is the work around to make deja-dup work in the GUI.

Revision history for this message
Jim Caughran (caughranjim) wrote : Re: [Bug 1217959] Re: 'metadata' file not found when creating backup ("Could not restore ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"

I gave up on it. I wanted to put .cache into RAM storage, and creating a
backup there was just too much.

I use simplebackup, which is quite good.

===========
Jim Caughran
<email address hidden>

On 24 October 2014 11:34, Nathaniel Homier <email address hidden>
wrote:

> What is the work around to make deja-dup work in the GUI.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
Dabo Ross (daboross) wrote :

This is happening to me after after upgrading to 14.04, this worked fine before in 12.04. I have ~/.cache/deja-dup symlinked to /media/external/deja-dup-cache because the cache always gets too large to fit onto my home partition. /media/external/ is the external hard drive that I back up to.

If I remove the symlink the backup works, but it uses practically all of my home partition space.

Revision history for this message
arne (ubuntu-stukjewebgebeuren) wrote :

Happing to me too. I also have .cache/deja-dup symlinked to elsewhere, in my case to a folder on a extra hard drive. When I change the symlink back to a regular folder, backup runs fine.

Revision history for this message
Brandon P. (brandonmpace) wrote :

After looking at the code, I would prefer an option to specify the path to the deja-dup metadata file. (at least a config file edit, if not a GUI addition)

It seems to me it is coded to stick to the user's cache directory. (and duplicity may not be handling the symlink correctly)

Revision history for this message
Mark Preston (emarkpreston) wrote :

This has never happened to me before 1-January-2015. I run a weekly backup. Today, Deja told me it was doing a complete backup to prevent database corruption and that it would take longer than usual.

I do not have a power saving mode in use on this desktop 'puter. If I go away from it, I turn the computer's monitor off manually. There is no sleep or hibernate configged for this system. Except at cold boot time. If the password isn't entered before a set time, the password log-in screen/page shown on the monitor goes black and the mouse or keyboard key must be moved to bring the power back to the screen. I'ld turn that off too, if I knew how to.

Deja-Dup gets called every Thursday per it's config. After the external backup hard disk drive is powered up, Deja come up, asks for a password then runs/makes a backup. To shut down, from a terminal I always do: sync;sync;sync; - wait for the command line prompt to re-appear, then from the panel icon, I "unmount". This is invariably how I operate Deja-Dup.

Today, I did turn the monitor off while the deja-dup was running. I cannot see how that could cause this problem. Looking at the files made by deja-dup, which are stored on an external sata2 drive, I can find some file dates for today 1-January-2015. That is all I know.

Revision history for this message
David Janke (cppege430dtvg7d94ro-david-9ei9nyjpwdexk1if796) wrote :

Was having the same problem... ~/.cache was a symlink to a folder on another drive

Moved .cache into home directory (i.e., got rid of symlink and copied files to main drive) and the error went away

Revision history for this message
Kieran Boyce (kieran-boyce) wrote :

Affects me also since I synlinked my home directory to another hard drive over the holidays. Ubuntu 14.04 LTS 64-bit

Revision history for this message
Mark Preston (emarkpreston) wrote :

In regard to #41 of mine, above, I report that today 16 Jan 2015, Deja ran, completed it's backup and gave no error messages upon completion. Deja closed down and exited AFAIK.

Revision history for this message
Charles (slivkoff) wrote :

#34 is my problem.

My passwd entry is listed as /home/username, but /home is a sym-link to /Volumes/storage/home.

I am going to try changing my passwd entry to point to the true path as a workaround.

Revision history for this message
cement_head (andorjkiss) wrote :

Also have this problem after upgrading from 12.04 to 14.04

Could not restore ‘/home/<name>/.cache/deja-dup/metadata’: File not found in backup

Revision history for this message
HX_unbanned (linards-liepins) wrote :

Same issue as Comment #46 on Fedora 21 xb6_64.

Revision history for this message
tr33m4n (tr33m4n) wrote :

Also experiencing the "Could not restore ‘/home/<name>/.cache/deja-dup/metadata’: File not found in backup" problem in 15.04 when trying to backup to an external hard disk

Revision history for this message
Max Ehrlich (queuecumber) wrote :

Ubuntu 15.04 deja-dup 34.0 also having this issue

Revision history for this message
Rodrigo Virote Kassick (kassick) wrote :

Seen on an up-to-date ubuntu 15.04

Removed the DejaDup line from accels and restarted nautilus, worked ok even after a reboot -- not sure yet if it will last

Revision history for this message
Rainer Defender (materialdefender2306) wrote :

Another "me also" (because of symlinked ~/.cache). I would like to ask if in the meantime, while this is being looked into, we could get the information as to the duplicity command ran by Deja-Dup so we can just run that manually until Duplicity is fine again.

Revision history for this message
Rainer Defender (materialdefender2306) wrote :

I.e., not the abstract command as it can be looked up in the sources, but the actual command - there *must* be a way to get to that so it can be copied & pasted - right?

Revision history for this message
Buz Wilson (buzwilson) wrote :

I fixed this error by simply making a metadata file in .cache/deja-dup
i.e., from the home directory issue the terminal command

           touch .cache/deja-dup/metadata

dejadup-preferences is still throwing CRITICAL errors but probably unrelated to this

Revision history for this message
Jim Baker (jwb45) wrote :

I am running Linux Mint 17.1 and got this error after the first backup. If anyone is still monitoring this, what is the best solution. Or do I just remove deja-dup and move on. Error is: "Could not restore ‘/home/<name>/.cache/deja-dup/metadata’: File not found in backup". I am backing up to a USB connected hard drive.

Revision history for this message
Kim Holder (kimholder) wrote :

 Ubuntu 14.04 64 bit, kernel 3.16.0-30-generic

I have gotten this error several times, backing up to a usb hard drive. There were no interruptions to the backup. It gets to the end, the step of validating the backup, and then throws up this error message.

Revision history for this message
Kirk Erickson (ekirk0) wrote :
Download full text (5.5 KiB)

Getting the "metadata not found in archive, no files restored" error

I have a ssd primary drive with a soft symlink to another drive for all home folders.
The ~/.cache/deja-dup/metadata exists on that second drive.

Questions:

For duplicity does archiving a folder under its symlinked and real location cause clashing?
Does duplicity it back it up twice?
Is duplicity still executing when deja-dup goes to delete the metadata folder. Since this is async processing in deja.

Test:

Ran deja from commad line to reproduce error. Here is how to get the command args and errors from deja.

export DEJA_DUP_DEBUG=1
deja-dup --backup > dump

The folder to be archived (/media/storage/home/kirk/) is the actual location where my $home points to
only see one include.

DUPLICITY: . Args: /usr/bin/duplicity
--exclude=/media/storage/backup
--exclude=/media/storage/home/kirk/Downloads
--exclude=/media/storage/home/kirk/.local/share/Trash
--exclude=/media/storage/home/kirk/.cache/deja-dup/tmp
--exclude=/media/storage/home/kirk/.xsession-errors
--exclude=/media/storage/home/kirk/.thumbnails
--exclude=/media/storage/home/kirk/.adobe/Flash_Player/AssetCache
--exclude=/media/storage/home/kirk/.cache/deja-dup
--exclude=/media/storage/home/kirk/.cache
--include=/media/storage/home/kirk
--include=/etc
--exclude=/home/kirk/Downloads
--exclude=/home/kirk/.local/share/Trash
--exclude=/sys
--exclude=/run
--exclude=/proc
--exclude=/home/kirk/.cache/deja-dup/tmp
--exclude=/var/tmp
--exclude=/tmp
--exclude=/home/kirk/.xsession-errors
--exclude=/home/kirk/.thumbnails
--exclude=/home/kirk/.steam/root
--exclude=/home/kirk/.recently-used.xbel
--exclude=/home/kirk/.recent-applications.xbel
--exclude=/home/kirk/.Private
--exclude=/home/kirk/.gvfs
--exclude=/home/kirk/.adobe/Flash_Player/AssetCache
--exclude=/home/kirk/.cache/deja-dup
--exclude=/home/kirk/.cache
--exclude=** --gio --dry-run --volsize=50 / file:///media/storage/backup --no-encryption --verbosity=9 --gpg-options=--no-use-agent --archive-dir=/home/kirk/.cache/deja-dup --tempdir=/tmp --log-fd=15

The folder to be archived is just the $home which points to (/media/storage/home/kirk/)
can see two includes which point to the same drive location

DUPLICITY: . Args: /usr/bin/duplicity
--exclude=/media/storage/backup --include=/etc
--exclude=/home/kirk/Downloads
--exclude=/home/kirk/.local/share/Trash
--exclude=/home/kirk/.cache/deja-dup/tmp
--exclude=/home/kirk/.xsession-errors
--exclude=/home/kirk/.thumbnails
--exclude=/home/kirk/.steam/root
--exclude=/home/kirk/.recently-used.xbel
--exclude=/home/kirk/.recent-applications.xbel
--exclude=/home/kirk/.Private
--exclude=/home/kirk/.gvfs
--exclude=/home/kirk/.adobe/Flash_Player/AssetCache
--exclude=/home/kirk/.cache/deja-dup
--exclude=/home/kirk/.cache
--include=/home/kirk
--exclude=/media/storage/home/kirk/Downloads
--exclude=/media/storage/home/kirk/.local/share/Trash
--exclude=/media/storage/home/kirk/.cache/deja-dup/tmp
--exclude=/media/storage/home/kirk/.xsession-errors
--exclude=/media/storage/home/kirk/.thumbnails
--exclude=/media/storage/home/kirk/.adobe/Flash_Player/AssetCache
--exclude=/media/storage/home/kirk/.cache/deja-dup
--exclude=/media/...

Read more...

Revision history for this message
Simon Déziel (sdeziel) wrote :

In my case, the problem was caused by tighter permissions on "/home". I had it set to "0711" which apparently deja-dup is allergic to. chmod'ing it to the more common "0755" made it work again.

Revision history for this message
Rob Frohne (frohro) wrote :

I too have this problem on the first backup with ~/.cache/deja-dup symbolically linked as follows:

lrwxrwxrwx 1 frohro frohro 33 Oct 7 08:46 deja-dup -> /mnt/frohro/.cache/deja-dup-cache

I'm going to try excluding /mnt/frohro/.cache/deja-dup-cache. I will post results.

Revision history for this message
Kirk Erickson (ekirk0) wrote :

What does duplicity report as backup status?

duplicity collection-status file:///[path to your backup folder]

On Thu, Oct 8, 2015 at 8:54 AM, Rob Frohne <email address hidden> wrote:

> I too have this problem on the first backup with ~/.cache/deja-dup
> symbolically linked as follows:
>
> lrwxrwxrwx 1 frohro frohro 33 Oct 7 08:46 deja-dup ->
> /mnt/frohro/.cache/deja-dup-cache
>
> I'm going to try excluding /mnt/frohro/.cache/deja-dup-cache. I will
> post results.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
Rob Frohne (frohro) wrote :

Hi Kirk,

Here is what I get:

frohro@frohro-e6420:~/.cache$ duplicity collection-status file:///media/frohro/Backups/Rob/
Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20151008T005844Z.sigtar.gz to local cache.
Copying duplicity-full.20151008T005844Z.manifest to local cache.
Last full backup date: Wed Oct 7 17:58:44 2015
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive dir: /home/frohro/.cache/duplicity/a051959fc7c263d30b8ce86da1b779e2

Found 0 secondary backup chains.

Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed Oct 7 17:58:44 2015
Chain end time: Wed Oct 7 17:58:44 2015
Number of contained backup sets: 1
Total number of contained volumes: 9499
 Type of backup set: Time: Num volumes:
                Full Wed Oct 7 17:58:44 2015 9499
-------------------------
No orphaned or incomplete backup sets found.

What does that tell you?

Thanks,

Rob

Revision history for this message
Kirk Erickson (ekirk0) wrote :

Looks like a good report.

Need to look into restoring from these backups. duplicity
<http://duplicity.nongnu.org/>

http://askubuntu.com/questions/520152/extract-a-folder-from-duplicity-archive

Looks like you can test extracting a folder/file with duplicity.

On Thu, Oct 8, 2015 at 11:34 AM, Rob Frohne <email address hidden> wrote:

> Hi Kirk,
>
> Here is what I get:
>
> frohro@frohro-e6420:~/.cache$ duplicity collection-status
> file:///media/frohro/Backups/Rob/
> Synchronizing remote metadata to local cache...
> Copying duplicity-full-signatures.20151008T005844Z.sigtar.gz to local
> cache.
> Copying duplicity-full.20151008T005844Z.manifest to local cache.
> Last full backup date: Wed Oct 7 17:58:44 2015
> Collection Status
> -----------------
> Connecting with backend: BackendWrapper
> Archive dir: /home/frohro/.cache/duplicity/a051959fc7c263d30b8ce86da1b779e2
>
> Found 0 secondary backup chains.
>
> Found primary backup chain with matching signature chain:
> -------------------------
> Chain start time: Wed Oct 7 17:58:44 2015
> Chain end time: Wed Oct 7 17:58:44 2015
> Number of contained backup sets: 1
> Total number of contained volumes: 9499
> Type of backup set: Time: Num volumes:
> Full Wed Oct 7 17:58:44 2015 9499
> -------------------------
> No orphaned or incomplete backup sets found.
>
> What does that tell you?
>
> Thanks,
>
> Rob
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
Rob Frohne (frohro) wrote :

Well, I tried to use duplicity to extract a file and it wasn't found. I then tried to list-current-files and got:

frohro@frohro-e6420:~$ duplicity list-current-files file:///home/frohro
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1500, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1494, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1343, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1432, in do_backup
    list_current(col_stats)
  File "/usr/bin/duplicity", line 665, in list_current
    sig_chain = col_stats.get_signature_chain_at_time(time)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 979, in get_signature_chain_at_time
    raise CollectionsError("No signature chains found")
CollectionsError: No signature chains found

frohro@frohro-e6420:~$

I think something is amiss. I'm going to try to backup again tonight and see what happens with the symbolically linked deja-dup-cache file excluded.

Rob

Revision history for this message
cement_head (andorjkiss) wrote :

Weird, I just made an encryped backup and this error jumped at me. Is there a solution?

Ubuntu 15.10 running DejaDup to an external USB 3.0 HDD.

Thanks,
Andor

Revision history for this message
Kirk Erickson (ekirk0) wrote :

no solution yet. i believe the backups are good. add your comments to
this bug to push things along.

On Mon, Mar 7, 2016 at 3:05 PM, cement_head <email address hidden>
wrote:

> Weird, I just made an encryped backup and this error jumped at me. Is
> there a solution?
>
> Ubuntu 15.10 running DejaDup to an external USB 3.0 HDD.
>
> Thanks,
> Andor
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
cement_head (andorjkiss) wrote :

Will creating an empty "metadata" file cure this issue? I also think that the backups are good.

Revision history for this message
Kirk Erickson (ekirk0) wrote :

no creating the metadata file, deleting the config folder and rebooting do
not work.

deja dup simply calls duplicity script http://duplicity.nongnu.org/ to do
the actual backup. try just using the duplicty script in a cronjob if you
need an immediate.

my guess: the gui dejadup is failing due to some kind of asyn processing
hickup. metadata check trick is evaluated too soon and clobbers the
duplicity finalize step. vala engineer needed to fix this defect.

On Tue, Mar 8, 2016 at 8:07 AM, cement_head <email address hidden>
wrote:

> Will creating an empty "metadata" file cure this issue? I also think
> that the backups are good.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
cement_head (andorjkiss) wrote :

Okay. Created an "metadata" file last week. Automatic backup stared via the DejaDup GUI, completed, prompted for password, tested backup, everything A-OK.

Perhaps the **first** instance there's a sequential execution bug and DejaDup generates an error?

- Andor

Xytime (xytime)
no longer affects: deja-dup (Arch Linux)
Revision history for this message
Marius Nuennerich (mwrius) wrote :

Still happening on 16.04.

I *DON'T* have .cache symlinked anywhere. I triggered this by manually starting a backup through the preferences window. I have Deja-Dup configured to run daily and a few hours ago the automatic run for the day already ran.

Revision history for this message
Leon Arundell (leon-arundell) wrote :

Please upgrade the status of this bug to "High."

It has occurred several times on my Ubuntu 14.04 system in the past two months, and means that I cannot reliably use Duplicity to do incremental backups. The main alternatives seem to only offer time- and space- consuming complete backups.

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

Go here <https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa> and add
the repository to your list. apt-get update / upgrade will bring in the
latest version which is a lot newer than the one you have.

0.6.21 is at least 3 years old and has been superseded.

On Wed, Jun 1, 2016 at 3:46 AM, Leon Arundell <email address hidden>
wrote:

> Please upgrade the status of this bug to "High."
>
> It has occurred several times on my Ubuntu 14.04 system in the past two
> months, and means that I cannot reliably use Duplicity to do incremental
> backups. The main alternatives seem to only offer time- and space-
> consuming complete backups.
>
> --
> You received this bug notification because you are subscribed to
> duplicity in Ubuntu.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
> Status in duplicity package in Ubuntu:
> New
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
Leon Arundell (leon-arundell) wrote :

I tried Rob Frohne's suggestion of adding ~/.cache.deja-dup to my "Folders to ignore" list, and it has worked at least temporarily.

I also tried Kenneth Loafman's suggestion of updating duplicity, but the installation instructions didn't seem to work.

Revision history for this message
cement_head (andorjkiss) wrote :

@leon-arundell What do you mean by "I also tried Kenneth Loafman's suggestion of updating duplicity, but the installation instructions didn't seem to work."?

Do you mean to say that you couldn't add PPA, update the cache and upgrade Duplicity? Or do you mean that after upgrading Duplicity, there was no change?

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in duplicity (Ubuntu):
status: New → Confirmed
Changed in duplicity (Ubuntu):
status: Confirmed → Incomplete
Revision history for this message
novoid (v-launchpad-karl-voit-at) wrote :

I am affected by this bug with a current Xubuntu 16.04 LTS.

My ~/.cache is a symlink to a different SSD/partition if this matters. Since ~/.cache/deja-dup filled up my root parition (before symlinking it), I had to manually delete ~/.cache/deja-dup and the files in the backup destination. Maybe this had some effect as well.

Revision history for this message
Mike Gerber (mikegerber) wrote :

I am seeing this on Ubuntu 16.10. ~/.cache/deja-dup/ is on the /home filesystem.

$ dpkg -l duplicity deja-dup
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii deja-dup 34.2-0ubuntu amd64 Back up your files
ii duplicity 0.7.06-2ubun amd64 encrypted bandwidth-efficient bac

Revision history for this message
Mike Gerber (mikegerber) wrote :

$ ls ~/.cache/deja-dup/
4d801e19c0c7ed7e0ced1c98fe748294 tmp

Revision history for this message
Felipe Castillo (fcastillo.ec) wrote :

I have this exact same problem. I tried creating an empty metadata file so it can write over it, but that didn't work.
I decided to symlink only the folder inside '.cache/deja-dup', the folder named 6724d3f170e0390931977a130c2b7632. I thought by leaving the deja-dup folder, it will be able to write the metadata file but still have everything else symlink somewhere else. It didn't work, and it was nasty. As soon as I tried to ran a manual backup (from the GUI, of course) I saw this error:

"Specified archive directory '/home/[user]/.cache/deja-dup/6724d3f170e0390931977a130c2b7632' does not exist, or is not a directory"

Well, I guess it's technically not a directory, it's a symlink to one.

Has anybody come up with a workaround for this? Anything?

Revision history for this message
Dabo Ross (daboross) wrote :

I've worked around this by using a bind mount instead of a symlink. Something like 'sudo mount --rbind /run/media/daboross/external/deja-dup-cache ~/.cache/deja-dup' works. While I do have to re-do this every time I mount the external drive and start a backup, it at least works.

If you also do a `chmod 500 ~/.cache/deja-dup` when it's unmounted, deja-dup will be unable to write any files to the unmounted directory and it will fail early - so you can then mount it and re-start the backup.

Revision history for this message
Xwarman (xwarman) wrote :

That bug is now over 3 years old and still existent. In the default backup software of ubuntu... I... am just wondering.

Well. Can I still use it and are the backups usable anyway? Can I trust them with that error?

Thanks.

Revision history for this message
cement_head (andorjkiss) wrote :

See Post#77 for the workaround, and yes, this will make it all work

Revision history for this message
Vej (vej) wrote :

@Xwarman: Please be aware that you disabled one of the major checks to ensure a successful backup.

@all: I am wondering if this happens before or after the backup taking place?

A more recent output of the file /tmp/deja-dup.log after running the appropriate line below and replicating the problem might be helpful as well. Especially because there seems to be another problem in the backup of the original reporter. You may want to scrub the log of any incriminating file names or details:

    DEJA_DUP_DEBUG=1 deja-dup --backup | tail -n 1000 > /tmp/deja-dup.log

Revision history for this message
Vej (vej) wrote :

Setting this to Incomplete, because I am unable to verify without the information requested above.

Please reset to "Confirmed" after providing this information.

Changed in deja-dup:
status: Confirmed → Incomplete
Revision history for this message
Dabo Ross (daboross) wrote :

When this last occurred for me, it happened after the backup was complete, or at least after the backup had been running for a while. I'm unsure if it was during, before or after the 'verification' phase.

Just now, when running the command you posted, the error did not occur. I haven't had this error anyways for quite some time since I've been using the workaround of having an rbind mount, but just now I re-did the symlink which caused the error before, and the error did not occur. I'm now using deja-dup 34.2 in Gentoo, and when it originally occurred I was using an earlier version of deja-dup in Ubuntu. I'm not sure if it's completely fixed for me, if my different setup can no longer reproduce the error, or if it just worked this one time, so if anyone else who had experienced in the past could test with deja-dup 34.2, that would probably be best.

Revision history for this message
Ricardo Graça (devius) wrote :

@Vej I just saw this error this morning, but on my second attempt using the command you provided to get a log file there was no error. However, upon inspection of the log file I can see that it didn't process all "volumes". It was supposed to process 258 volumes, but the log file ends on volume 11. I have no idea what these "volumes" are, and they may be called something else in the English version. I'm using the Portuguese version.

Do you still want me to provide the log file?

Revision history for this message
Vej (vej) wrote :

@Ricardo If your log does still contain an line "DUPLICITY: ERROR 19" it might be helpful here. Otherwise I would think, that you are facing another bug.

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for Déjà Dup because there has been no activity for 60 days.]

Changed in deja-dup:
status: Incomplete → Expired
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for duplicity (Ubuntu) because there has been no activity for 60 days.]

Changed in duplicity (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Dabo Ross (daboross) wrote :

I thought this had been fixed, but I ran into something else very similar today:

When I had a symlink set for ~/.cache/deja-dup, deja-dup actually _deleted_ this symlink, and still filled up my home folder. I ran out of space and the backup failed.

Perhaps this is why it was succeeding for me, just by coincidence I had enough space at the time?

Or would it be better to make a new issue for this seeming deletion of symbolic links?

deja-dup 34.3

Revision history for this message
Vej (vej) wrote :

@Dabo Ross: Your problem seems to be different from this one, so a new bug report is a good idea.

Btw: Please refrain from exaggerating "a symlink set for ~/.cache/deja-dup gets deleted" to "deletion of symbolic links", because the handling of ~/.cache/deja-dup is a bit special (unless you see deletion of other symlinks as well).

Thanks

Vej

Revision history for this message
Dabo Ross (daboross) wrote :

Alright, thank you. My apologies for the accidentally inflating the issue, I was not thinking through my comment much when I posted.

Revision history for this message
Michael Terry (mterry) wrote :

Reopening. I can easily believe we don't handle a symlinked cache file well.

Changed in deja-dup:
status: Expired → New
Revision history for this message
Jared Stemen (jstemen) wrote :

I just hit this bug. I recently put in a symlink for my ~/.cache directory. My deja dup version is as follows:

~/.cache$ apt list --installed | grep deja

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

deja-dup/xenial-updates,now 34.2-0ubuntu1.1 amd64 [installed]
deja-dup-backend-gvfs/xenial-updates,xenial-updates,now 34.2-0ubuntu1.1 all [installed,automatic]
deja-dup-backend-s3/xenial-updates,xenial-updates,now 34.2-0ubuntu1.1 all [installed]
fonts-dejavu/xenial,xenial,now 2.35-1 all [installed,automatic]
fonts-dejavu-core/xenial,xenial,now 2.35-1 all [installed]
fonts-dejavu-extra/xenial,xenial,now 2.35-1 all [installed,automatic]

Revision history for this message
Vej (vej) wrote :

Hello Jared.

Thanks for letting us know about this.

Can you please add the file /tmp/deja-dup.log after running the line below and replicating the problem ? This is an old bug so a new log might be helpful for us to figure out what went wrong here:

    DEJA_DUP_DEBUG=1 deja-dup --backup | tail -n 1000 > /tmp/deja-dup.log

Best Regards

Vej

Changed in duplicity (Ubuntu):
status: Expired → Incomplete
Changed in deja-dup:
status: New → Incomplete
Revision history for this message
Tuomo Sipola (tuomosipola) wrote :

Still affects me. I have symlinked Deja Dup cache deja-dup -> /media/username/Space/.deja-dup-cache

Revision history for this message
Marcin Owsiany (marcin-owsiany-pl) wrote :

vej, it affects me as well, using 34.3-1

Attaching the log.

In there one can see:

DUPLICITY: ERROR 19
DUPLICITY: . home/porridge/.cache/deja-dup/metadata not found in archive - no files restored.

And once I think about it, it makes sense, since /home is a symlink to /srv/home, and the metadata directory is there under the canonical path:

porridge@backup-server$ duplicity list-current-files file://`pwd`|grep cache/deja-dup
GnuPG passphrase for decryption:
Thu Mar 1 17:16:19 2018 srv/home/porridge/.cache/deja-dup
Thu Mar 1 17:16:19 2018 srv/home/porridge/.cache/deja-dup/metadata
Thu Mar 1 17:16:19 2018 srv/home/porridge/.cache/deja-dup/metadata/README
porridge@backup-server$

How can I work around this?

Changed in deja-dup:
status: Incomplete → Confirmed
Revision history for this message
Marcin Owsiany (marcin-owsiany-pl) wrote :

I'm wondering whether something like wrapping the Path.build_filename call located in libdeja/OperationVerify.vala:connect_to_job in a realpath() would be enough to fix this issue.

Unfortunately I don't know vala well enough to test this idea.

Revision history for this message
Kirk Erickson (ekirk0) wrote :

Deja calls the duplicity command. Monitors the progress and checks that it
finishes. Try getting the underlying command and manually calling
duplicity. When I last looked at this bug duplicity was performing the
backup job to completion. Backups we're ok. Just this error coming from
deja. After looking at the vala code it looked like a vala async call was
failing.

*Check your backups. Do they work?
*Try the deja debug mode to get logs.

export DEJA_DUP_DEBUG=1
deja-dup --backup > dump

Maybe this helps

On Mar 1, 2018 1:19 PM, "Marcin Owsiany" <email address hidden> wrote:

I'm wondering whether something like wrapping the Path.build_filename
call located in libdeja/OperationVerify.vala:connect_to_job in a
realpath() would be enough to fix this issue.

Unfortunately I don't know vala well enough to test this idea.

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1217959

Title:
  'metadata' file not found when creating backup ("Could not restore
  ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"

Status in Déjà Dup:
  Confirmed
Status in duplicity package in Ubuntu:
  Incomplete

Bug description:
  Linux release: Ubuntu 13.04 64bit
  Kernel: 3.8.0-29-generic
  Python: 2.7.4
  Deja Dup: 26.0
  Duplicity: 0.6.21

  Hi. I've tried to do my monthly or so backup recently, but Deja Dup
  crashed after trying to backup a very large file (VM disk with Windows
  7, 50GB or so). I tried to do another one, this time excluding all my
  large VM's. However, now when trying to do a backup I'm getting the
  following error a few seconds after inputting my encryption password:

  Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
  in backup

  I've tried reinstalling both deja-dup and duplicity, purging
  duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
  deleting the backup files on the external drive, deleting it's config
  as described here http://askubuntu.com/questions/53980/how-to-delete-
  all-the-settings-for-deja-dup. I know (well, I have some evidence
  supporting this) that the drive itself isn't faulty as I managed to
  backup my old laptop to the drive using duplicity and it worked fine
  (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
  momentarily.

To manage notifications about this bug go to:
https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions

Revision history for this message
Marcin Owsiany (marcin-owsiany-pl) wrote :

Kirk, I think you missed my update #95, which explains the root cause.

Revision history for this message
Vej (vej) wrote :

@Marcin For a workaround you can try mounting with --bind instead of symlinking.

If you do, please let us now if this worked for you.

Best

Vej

Revision history for this message
Marcin Owsiany (marcin-owsiany-pl) wrote :

I tend to avoid bind-mounts because they confuse disk usage calculation
programs.
What I ended up doing was use "usermod" to set the home to canonical path.
deja-dup works now, and surprisingly I haven't notice any other app break
(so far).

2018-03-04 21:18 GMT+01:00 Vej <email address hidden>:

> @Marcin For a workaround you can try mounting with --bind instead of
> symlinking.
>
> If you do, please let us now if this worked for you.
>
> Best
>
> Vej
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1217959
>
> Title:
> 'metadata' file not found when creating backup ("Could not restore
> ‘/home/user /.cache/deja-dup/metadata’: File not found in backup"
>
> Status in Déjà Dup:
> Confirmed
> Status in duplicity package in Ubuntu:
> Incomplete
>
> Bug description:
> Linux release: Ubuntu 13.04 64bit
> Kernel: 3.8.0-29-generic
> Python: 2.7.4
> Deja Dup: 26.0
> Duplicity: 0.6.21
>
> Hi. I've tried to do my monthly or so backup recently, but Deja Dup
> crashed after trying to backup a very large file (VM disk with Windows
> 7, 50GB or so). I tried to do another one, this time excluding all my
> large VM's. However, now when trying to do a backup I'm getting the
> following error a few seconds after inputting my encryption password:
>
> Could not restore ‘/home/tom/.cache/deja-dup/metadata’: File not found
> in backup
>
> I've tried reinstalling both deja-dup and duplicity, purging
> duplicity, deleting deja-dup and duplicity from /home/tom/.cache,
> deleting the backup files on the external drive, deleting it's config
> as described here http://askubuntu.com/questions/53980/how-to-delete-
> all-the-settings-for-deja-dup. I know (well, I have some evidence
> supporting this) that the drive itself isn't faulty as I managed to
> backup my old laptop to the drive using duplicity and it worked fine
> (Fedora 19, deja dup 26.0 I think). I will upload the appropriate logs
> momentarily.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/deja-dup/+bug/1217959/+subscriptions
>

Revision history for this message
Michael Terry (mterry) wrote :

Sorry for ignoring this for so long! I've got a patch in master for this and a few related symlink-restore issues. Fix should be in the 38.3 release.

https://git.launchpad.net/deja-dup/commit/?id=569ec2b552122025c4636f95fd956df965f70c9b

Changed in deja-dup:
status: Confirmed → Fix Committed
Michael Terry (mterry)
Changed in deja-dup:
status: Fix Committed → Fix Released
Revision history for this message
Sergey Shik (pjss) wrote :

Updated to latest version but still got that error
ii deja-dup 39~bzr1796~u amd64 Back up your files
ii duplicity 0.7.17-0ubun amd64 encrypted bandwidth-efficient bac

my .cache/deja-dup symlinked to another folder

P.S. is it possible to include cache location option in preference?

Revision history for this message
Bas Roufs (basroufs) wrote :

Working at a fresh install of Kubuntu 18.04.1 LTS in February 2019 - at a Lenovo X220, with a i5 processor and a 1 TB Samsung 860 EVO SSD, together with a 4 TB external HD, "WD My Passport". However, I still stumble upon the same bug. However, I seemingly manage to work around it by acting as follows:
TERMINAL >> sudo apt-get purge deja-dup duplicity > sudo apt-get autoremove > sudo reboot now > after the reboot: TERMINAL >> apt-get install deja-dup duplicity > sudo deja-dup.

On one hand, deja-dup has been making a first back-up ever since I hit the "sudo-deja" commando - now, a few hours ago.

On the other hand, I got this terminal feedback in the first minutes after hitting the "sudo deja-dup" commanend.

bas@Viaconsensus-iter:~$ sudo deja-dup

** (org.gnome.DejaDup:2519): CRITICAL **: 14:34:51.229: string_contains: assertion 'self != NULL' failed
gpg: WARNING: unsafe ownership on homedir '/home/bas/.gnupg'

** (org.gnome.DejaDup:2519): WARNING **: 14:35:56.153: AssistantOperation.vala:904: Error calling StartServiceByName for org.freedesktop.secrets: Timeout was reached

After "Timeout was reached", deja-dup seemingly started the backup process that is still ongoing.

I keep you posted here on my experiences.

Yours,
Bas G. ROufs.

Revision history for this message
Bas Roufs (basroufs) wrote :
Revision history for this message
Bas Roufs (basroufs) wrote :
Revision history for this message
Savvas Shiakas (shiakas) wrote :

Hey guys, I don't think this is a solution but it didn't produce the error so here it goes:
1. created a folder and named it "backups" in the home directory
2. changed the Storage location of the backup to the "backups" directory
3. created the backup and no error :)
Previously my storage location was the home directory.
(I am new to Ubuntu/Linux)

Revision history for this message
Kenneth Loafman (kenneth-loafman) wrote :

Please upgrade to the current version of duplicity. This will assure that any bugs fixed since your release are available and may fix your issue.

NOTE: This applies especially to duplicity versions between 0.7.03 and 0.7.14 inclusive. There was a fix in 0.7.15 that reduced memory usage drastically, and will help with memory errors and inability to start new threads.

There are three options:

* Release tarball Install - https://launchpad.net/duplicity/+download
* Daily duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/daily
* Stable duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/ppa

NOTE: UNinstall duplicity first if it was installed via the distribution repository. For Ubuntu, that would be "sudo apt-get purge duplicity".

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.