.mozilla and .thunderbird not backed up

[Michael Terry]

Hello!

Ubuntu Mate 16.04.1 - completely up to date.

I recently did a full back up of my primary user id /home/user directory. When restoring the directory I noticed immediately that several very important items were missing.

All of my desktop config and preferences were gone. I don't know what files they were contained in. Also, the .Mozilla and .Thunderbird directories were never backed up and of course could not be restored. I have since done the restore again to a separate directory for inspection. They are not there. Why were they not backed up when I explicitly told it to back up everything? There were no errors on backup or restore.

I have since done the full backup again (unencrypted). Sure as H.E.L.L. it will NOT backup those files and directories.

What is the point of a back up application that will not back up configurations, passwords. bookmarks, mail?

I don't mean to be disrespectful to those of you who volunteer to make this app available. I am mad as all get out because I lost nearly everything that takes my time to rebuild. That is why I backed it up.

[Naël (nathanael-naeri)]

Uhhh... Michael? Aren't you the one who developed Déjà-Dup in the first place? Now I'm confused...

This issue does sound bad though.

[Naël (nathanael-naeri)]

Oh, nevermind, I got it. Didn't see the link to the related question.

@youthinkso, I suppose the place to begin is inspecting a backup log? Hopefully that would help understanding what is going on. A backup log is produced with the command:

  DEJA_DUP_DEBUG=1 deja-dup --backup > backup.log

but it may be very large so perhaps pipe the output to grep instead, to look for interesting things, e.g.:

  DEJA_DUP_DEBUG=1 deja-dup --backup | grep '\.mozilla'
  DEJA_DUP_DEBUG=1 deja-dup --backup | grep '\.thunderbird'
  DEJA_DUP_DEBUG=1 deja-dup --backup | grep '\.config'

  DEJA_DUP_DEBUG=1 deja-dup --backup | grep -E '/usr/bin/duplicity| A '

Also, can you please attach your config of Déjà-Dup? Thanks.

  gsettings list-recursively org.gnome.DejaDup > deja-dup.config

[youthinkso (jlxb)]

nathanael-naeri, Well that is interesting. I ran DEJA_DUP_DEBUG=1 deja-dup --backup > backup.log presuming it would spin out a back up log. It actually did a back up and created a log of the new back up.

1) I found other directories ignored in the original backup /home/user/.config .cache .dbus .gconf .gnupg .local

2) When I ran from the command line.... not only did it see .mozilla and .thunderbird (recreated after disaster) it backed them both up with all others and had no issues. How do you like that?

3) The only difference was it was launched from a terminal and not from the desktop menu. But in doing so, it gave me zero options to tell it what to backup. It just flew in and sewed it up. It did however, follow generally the previous saved setup so far as I can tell.

see attached config file.

Thank you for all your assistance - much appreciated.

[Naël (nathanael-naeri)]

> I ran DEJA_DUP_DEBUG=1 deja-dup --backup > backup.log presuming
> it would spin out a back up log. It actually did a back up and
> created a log of the new back up.

Yes sorry I should have been more explicit: "deja-dup --backup" starts a backup, "DEJA_DUP_DEBUG=1" tells Déjà-Dup to output verbose debug information, and "> backup.log" redirects this output into a file.

> 1) I found other directories ignored in the original backup
> /home/user/.config .cache .dbus .gconf .gnupg .local

Certain dot-directories of $HOME are excluded by default, this is normal: ~/.gvfs, ~/.cache, ~/.cache/deja-dup, ~/.Private, ~/.adobe/Flash_Player/AssetCache, ~/.recent-applications.xbel, ~/.recently-used.xbel, ~/.steam/root, ~/.thumbnails, ~/.xsession-errors, /var/tmp, /tmp, /proc, /run, /sys.

It is not normal that ~/.config, ~/.dbus, ~/.gconf, ~/.gnupg, ~/.local are excluded, unless you specifically asked for them to not be backed up by adequately setting up your include-list and exclude-list. In particular, ~/.config is where most programs store their configuration, and it should be backed up in the default configuration (include-list=['$HOME'], exclude-list=['$TRASH', '$DOWNLOAD']).

If you can provide a backup log that shows these directories being excluded (i.e. passed as --exclude options to the backup software, duplicity), I'd love to have a look at it. Feel free to anonymize whatever information you want in this log.

Without such a log file however, there's unfortunately not much we can do to troubleshot forward.

> 2) When I ran from the command line.... not only did it see
> .mozilla and .thunderbird (recreated after disaster) it backed
> them both up with all others and had no issues. How do you like
> that?

Better. This is the indented behavior, and the one I can enjoy on my machine.

> 3) The only difference was it was launched from a terminal and
> not from the desktop menu.

It isn't supposed to make any difference whether you click "Backup Now" in "System Settings > Backups" or run "deja-dup --backup", but I'll have a look to be sure.

> But in doing so, it gave me zero options to tell it what to backup.
> It just flew in and sewed it up. It did however, follow generally
> the previous saved setup so far as I can tell.

Yes clicking "Backup Now" or running "deja-dup --backup" both follow the settings you set up in "System Settings > Backups" (aka "deja-dup-preferences").

You printed these settings in a terminal window by running "gsettings list-recursively org.gnome.DejaDup" (redirected to a file with "> deja-dup.config"). They look fine to me. Just beware that you haven't set up a periodic backup (e.g. every week). You can do so even if your external disk isn't always connected. The backup will just be delayed until you connect it.

[Vej (vej)]

@youthinkso:

Are the affected directories still excluded in a regular backup, after your manual execution using Nathanaëls command?

I will set to "Incomplete", because the current informations do not allow to verify the problem. Please reset "Status" to "New" if anyone finds more information about this, or to "Confirmed" if someone else is affected by this bug.

[youthinkso (jlxb)]

"Are the affected directories still excluded in a regular backup, after
your manual execution using Nathanaëls command?"

No, they are no longer excluded.

Thanks for your assistance.

On 02/08/2017 06:50 AM, Vej wrote:
> @youthinkso:
>
> Are the affected directories still excluded in a regular backup, after
> your manual execution using Nathanaëls command?
>
> I will set to "Incomplete", because the current informations do not
> allow to verify the problem. Please reset "Status" to "New" if anyone
> finds more information about this, or to "Confirmed" if someone else is
> affected by this bug.
>
> ** Changed in: deja-dup
> Status: New => Incomplete
>

[Michael Terry (mterry)]

I'm closing this as no one can reproduce. I'm sorry it happened originally, not sure what would cause it. But glad it's no longer happening.