Restoring from new location does not find backup files

[ethanay]

[Impact]

When a user tries to restore a backup on a fresh Ubuntu install, deja-dup will not be able to find their backup at first.

A workaround is to set the current backup location on the fresh install to the old backup location. But that's not at all an obvious path. A user would not normally try that.

[Test Case]

1. Back up some files to a temporary dir.
2. gsettings reset-recursively org.gnome.DejaDup
3. Try to restore from that temporary dir.

Notice that deja-dup will complain about waiting for Google Drive to be set up (even though your backup drive has nothing to do with Google).

[Regression Potential]

The proposed patch mostly affects the restore dialog. Any regressions would likely be in that flow.

[Original Report]

1. I back up my computer to my external hard drive
2. I reinstall my new distro
3. I install deja-dup and duplicity
4. I open deja-dup, and select restore
5. I select my external hard drive and the restore directory
6. Deja-dup fails with "no backup found" message and creates a new backup directory on the hard drive

This indicated to me that deja dup ignored the restore directory I entered, and instead used the default directory, which did not exist, hence restore fails because the backend is told to look in the wrong place for the backup! This seems like more a user interface issue than an issue with the backup.

Here is the workflow I had to use to restore my backup:

1. Set storage location to "local folder" and then manually navigate to the folder on my external hard drive. If I did not do this specifically, then backup would fail. Selecting the external hard drive directly as the device and entering the folder in the "folder" field also fails.
2. Select restore and again use "local folder" and manually navigate to the folder on my external hard drive. If I instead try to use the hard drive device listed and enter the backup folder in the "folder" field, the restore also fails.

This issue occurs on both Ubuntu 18.04.1 and ElementaryOS Juno.

lsb_release -d
Description: elementary OS 5.0 Juno

dpkg-query -W deja-dup duplicity
deja-dup 37.1-2fakesync1
duplicity 0.7.17-0ubuntu1

gsettings list-recursively org.gnome.DejaDup > /tmp/deja-dup.gsettings
cat /tmp/deja-dup.gsettings
org.gnome.DejaDup last-restore '2018-11-23T01:39:44.556645Z'
org.gnome.DejaDup periodic true
org.gnome.DejaDup periodic-period 1
org.gnome.DejaDup full-backup-period 90
org.gnome.DejaDup backend 'local'
org.gnome.DejaDup last-run '2018-11-23T01:39:44.556645Z'
org.gnome.DejaDup nag-check ''
org.gnome.DejaDup prompt-check '2018-11-22T12:34:36.095106Z'
org.gnome.DejaDup root-prompt true
org.gnome.DejaDup include-list ['$HOME']
org.gnome.DejaDup exclude-list ['/home/ethan/.local/share/Trash']
org.gnome.DejaDup last-backup ''
org.gnome.DejaDup allow-metered false
org.gnome.DejaDup delete-after 0
org.gnome.DejaDup.Rackspace username ''
org.gnome.DejaDup.Rackspace container 'ethan-laptop'
org.gnome.DejaDup.S3 id ''
org.gnome.DejaDup.S3 bucket ''
org.gnome.DejaDup.S3 folder 'ethan-laptop'
org.gnome.DejaDup.OpenStack authurl ''
org.gnome.DejaDup.OpenStack tenant ''
org.gnome.DejaDup.OpenStack username ''
org.gnome.DejaDup.OpenStack container 'ethan-laptop'
org.gnome.DejaDup.GCS id ''
org.gnome.DejaDup.GCS bucket ''
org.gnome.DejaDup.GCS folder 'ethan-laptop'
org.gnome.DejaDup.Local folder '/media/ethan/porta_500gb/m1330'
org.gnome.DejaDup.Remote uri ''
org.gnome.DejaDup.Remote folder 'ethan-laptop'
org.gnome.DejaDup.Drive uuid '1919a422-d154-4e19-b551-f74b56cf1f0e'
org.gnome.DejaDup.Drive icon '. GThemedIcon drive-harddisk-usb drive-harddisk drive'
org.gnome.DejaDup.Drive folder 'm1330'
org.gnome.DejaDup.Drive name 'porta_500gb'
org.gnome.DejaDup.GOA id ''
org.gnome.DejaDup.GOA folder 'ethan-laptop'
org.gnome.DejaDup.GOA type 'google'
org.gnome.DejaDup.File short-name ''
org.gnome.DejaDup.File type 'normal'
org.gnome.DejaDup.File migrated true
org.gnome.DejaDup.File name ''
org.gnome.DejaDup.File path ''
org.gnome.DejaDup.File uuid ''
org.gnome.DejaDup.File icon ''
org.gnome.DejaDup.File relpath @ay []

[Michael Terry (mterry)]

Able to confirm, working on a fix. Marking as Critical, since it prevents restoring on a fresh system (a very common restoring use case).

There is a workaround (change current backup location settings to your old backup), but it's not an obvious one.

[ethanay (ethan-y-us)]

Conversely, another workaround is that you can rename/change your
current backup location to the Deja Dup default rather than trying to
get Deja Dup to recognize the current location. I haven't tested it but
that should also work.

On Sat, Jan 5, 2019 at 2:11 PM, Michael Terry <email address hidden> wrote:
> Able to confirm, working on a fix. Marking as Critical, since it
> prevents restoring on a fresh system (a very common restoring use
> case).
>
> There is a workaround (change current backup location settings to your
> old backup), but it's not an obvious one.
>
> ** Changed in: deja-dup
> Importance: Undecided => Critical
>
> --
> You received this bug notification because you are subscribed to the
> bug
> report.
> https://bugs.launchpad.net/bugs/1804744
>
> Title:
> restore from backup does not recognize custom backup directories
>
> Status in Déjà Dup:
> Confirmed
>
> Bug description:
> 1. I back up my computer to my external hard drive
> 2. I reinstall my new distro
> 3. I install deja-dup and duplicity
> 4. I open deja-dup, and select restore
> 5. I select my external hard drive and the restore directory
> 6. Deja-dup fails with "no backup found" message and creates a new
> backup directory on the hard drive
>
> This indicated to me that deja dup ignored the restore directory I
> entered, and instead used the default directory, which did not
> exist,
> hence restore fails because the backend is told to look in the wrong
> place for the backup! This seems like more a user interface issue
> than
> an issue with the backup.
>
> Here is the workflow I had to use to restore my backup:
>
> 1. Set storage location to "local folder" and then manually
> navigate to the folder on my external hard drive. If I did not do
> this specifically, then backup would fail. Selecting the external
> hard drive directly as the device and entering the folder in the
> "folder" field also fails.
> 2. Select restore and again use "local folder" and manually
> navigate to the folder on my external hard drive. If I instead try to
> use the hard drive device listed and enter the backup folder in the
> "folder" field, the restore also fails.
>
> This issue occurs on both Ubuntu 18.04.1 and ElementaryOS Juno.
>
> lsb_release -d
> Description: elementary OS 5.0 Juno
>
> dpkg-query -W deja-dup duplicity
> deja-dup 37.1-2fakesync1
> duplicity 0.7.17-0ubuntu1
>
> gsettings list-recursively org.gnome.DejaDup >
> /tmp/deja-dup.gsettings
> cat /tmp/deja-dup.gsettings
> org.gnome.DejaDup last-restore '2018-11-23T01:39:44.556645Z'
> org.gnome.DejaDup periodic true
> org.gnome.DejaDup periodic-period 1
> org.gnome.DejaDup full-backup-period 90
> org.gnome.DejaDup backend 'local'
> org.gnome.DejaDup last-run '2018-11-23T01:39:44.556645Z'
> org.gnome.DejaDup nag-check ''
> org.gnome.DejaDup prompt-check '2018-11-22T12:34:36.095106Z'
> org.gnome.DejaDup root-prompt true
> org.gnome.DejaDup include-list ['$HOME']
> org.gnome.DejaDup exclude-list ['/home/ethan/.local/share/Trash']
> org.gnome.DejaDup last-backup ''
> org.gnome.DejaDup allow-metered false
> org.gnome.DejaDup de...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jeb E. (jebeld17)]

I have this very same problem

On Sat, Jan 5, 2019, 10:25 AM Michael Terry <<email address hidden> wrote:

> ** Changed in: deja-dup
> Status: Fix Committed => Fix Released
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1788393).
> https://bugs.launchpad.net/bugs/1804744
>
> Title:
> Restoring from new location does not find backup files
>
> Status in Déjà Dup:
> Fix Released
> Status in deja-dup package in Ubuntu:
> Confirmed
>
> Bug description:
> 1. I back up my computer to my external hard drive
> 2. I reinstall my new distro
> 3. I install deja-dup and duplicity
> 4. I open deja-dup, and select restore
> 5. I select my external hard drive and the restore directory
> 6. Deja-dup fails with "no backup found" message and creates a new
> backup directory on the hard drive
>
> This indicated to me that deja dup ignored the restore directory I
> entered, and instead used the default directory, which did not exist,
> hence restore fails because the backend is told to look in the wrong
> place for the backup! This seems like more a user interface issue than
> an issue with the backup.
>
> Here is the workflow I had to use to restore my backup:
>
> 1. Set storage location to "local folder" and then manually navigate to
> the folder on my external hard drive. If I did not do this specifically,
> then backup would fail. Selecting the external hard drive directly as the
> device and entering the folder in the "folder" field also fails.
> 2. Select restore and again use "local folder" and manually navigate to
> the folder on my external hard drive. If I instead try to use the hard
> drive device listed and enter the backup folder in the "folder" field, the
> restore also fails.
>
> This issue occurs on both Ubuntu 18.04.1 and ElementaryOS Juno.
>
> lsb_release -d
> Description: elementary OS 5.0 Juno
>
> dpkg-query -W deja-dup duplicity
> deja-dup 37.1-2fakesync1
> duplicity 0.7.17-0ubuntu1
>
> gsettings list-recursively org.gnome.DejaDup > /tmp/deja-dup.gsettings
> cat /tmp/deja-dup.gsettings
> org.gnome.DejaDup last-restore '2018-11-23T01:39:44.556645Z'
> org.gnome.DejaDup periodic true
> org.gnome.DejaDup periodic-period 1
> org.gnome.DejaDup full-backup-period 90
> org.gnome.DejaDup backend 'local'
> org.gnome.DejaDup last-run '2018-11-23T01:39:44.556645Z'
> org.gnome.DejaDup nag-check ''
> org.gnome.DejaDup prompt-check '2018-11-22T12:34:36.095106Z'
> org.gnome.DejaDup root-prompt true
> org.gnome.DejaDup include-list ['$HOME']
> org.gnome.DejaDup exclude-list ['/home/ethan/.local/share/Trash']
> org.gnome.DejaDup last-backup ''
> org.gnome.DejaDup allow-metered false
> org.gnome.DejaDup delete-after 0
> org.gnome.DejaDup.Rackspace username ''
> org.gnome.DejaDup.Rackspace container 'ethan-laptop'
> org.gnome.DejaDup.S3 id ''
> org.gnome.DejaDup.S3 bucket ''
> org.gnome.DejaDup.S3 folder 'ethan-laptop'
> org.gnome.DejaDup.OpenStack authurl ''
> org.gnome.DejaDup.OpenStack tenant ''
> org.gnome.DejaDup.OpenStack username ''
> org.gnome.DejaDup.OpenStack container 'ethan-laptop'
> org.gno...

[Eero (eero+launchpad)]

Wow! It took 5 years from "Deja-Dup" to start working on this critical bug. I'd guess they don't even use their own application, because otherwise this had been an obvious show stopper bug a long time ago. Can Canonical/Ubuntu finally change the default backup application to something sane?

[ethanay (ethan-y-us)]

Eero I filed the original report. To me your comment seems only critical
and not helpful, especially in a report where a fix is being released. If
someone uses the same location for backup this problem doesn't appear to
exist.

I try to file bug reports as I encounter them, along with any workaround,
as a way of being helpful and contributing to something I believe in, they
are not always fixed but if you have an urgent issue that needs attention
you have plenty of options. Have you discussed on Ubuntu forums? filed a
separate report(s) for issues you face? posted a bounty? Use/purchase a
different program? Developing software takes money and/or time. It is
unfair as well as unhelpful to stand back and complain without contributing
either money or time to support. Asking people to work for free for the
most part is unreasonable. People deserve compensation for their work for
food, clothing, shelter, etc.

Apart from this interface bug the backup process has been fast and reliable
for me. I have relied on it on several occasions, and have yet to find a
program with the same power, reliability, flexibility and ease of use.

I use primarily Elementary OS which includes no backup software by default.
I don't like its default music player but that change to a preferred
software takes me less than a minute.

If you don't like deja dup an up-to-date forum post exploring/testing
preferred alternatives or outlining your specific unmet needs is more
helpful and potentially more constructive.

On Sun, Jan 6, 2019, 23:15 Eero <<email address hidden> wrote:

> Wow! It took 5 years from "Deja-Dup" to start working on this critical
> bug. I'd guess they don't even use their own application, because
> otherwise this had been an obvious show stopper bug a long time ago. Can
> Canonical/Ubuntu finally change the default backup application to
> something sane?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1804744
>
> Title:
> Restoring from new location does not find backup files
>
> Status in Déjà Dup:
> Fix Released
> Status in deja-dup package in Ubuntu:
> Confirmed
>
> Bug description:
> 1. I back up my computer to my external hard drive
> 2. I reinstall my new distro
> 3. I install deja-dup and duplicity
> 4. I open deja-dup, and select restore
> 5. I select my external hard drive and the restore directory
> 6. Deja-dup fails with "no backup found" message and creates a new
> backup directory on the hard drive
>
> This indicated to me that deja dup ignored the restore directory I
> entered, and instead used the default directory, which did not exist,
> hence restore fails because the backend is told to look in the wrong
> place for the backup! This seems like more a user interface issue than
> an issue with the backup.
>
> Here is the workflow I had to use to restore my backup:
>
> 1. Set storage location to "local folder" and then manually navigate to
> the folder on my external hard drive. If I did not do this specifically,
> then backup would fail. Selecting the external hard drive directly as the
> device and entering the folder in the "...

[Sebastien Bacher (seb128)]

It was fixed in https://git.launchpad.net/deja-dup/commit/?id=96f3d9ed and 38.2

[Sebastien Bacher (seb128)]

(seems also something worth SRUing to at least bionic)

[Launchpad Janitor (janitor)]

This bug was fixed in the package deja-dup - 38.2-1

---------------
deja-dup (38.2-1) unstable; urgency=medium

  * New upstream release:
    - Fix not being able to find the backup files when
      restoring on a fresh install (lp: #1804744)

 -- Sebastien Bacher <email address hidden> Fri, 11 Jan 2019 16:23:43 +0100

[Michael Terry (mterry)]

SRU patch for cosmic. I'll upload this shortly.

[Michael Terry (mterry)]

SRU debdiff for bionic.

[Michael Terry (mterry)]

I can't reproduce in xenial. So looks like this specific bug only affects bionic & cosmic still.

[Brian Murray (brian-murray)]

Hello ethanay, or anyone else affected,

Accepted deja-dup into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/deja-dup/38.0-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Brian Murray (brian-murray)]

Hello ethanay, or anyone else affected,

Accepted deja-dup into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/deja-dup/37.1-2fakesync1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[ethanay (ethan-y-us)]

We are traveling right now so I won't be able to test this for a few weeks.
Better if someone else can get to it sooner...thank you!

On Tue, Jan 15, 2019, 04:45 Brian Murray <<email address hidden> wrote:

> Hello ethanay, or anyone else affected,
>
> Accepted deja-dup into cosmic-proposed. The package will build now and
> be available at https://launchpad.net/ubuntu/+source/deja-
> dup/38.0-1ubuntu0.1 in a few hours, and then in the -proposed
> repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-cosmic to verification-done-cosmic. If it does not
> fix the bug for you, please add a comment stating that, and change the
> tag to verification-failed-cosmic. In either case, without details of
> your testing we will not be able to proceed.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> ** Changed in: deja-dup (Ubuntu Cosmic)
> Status: In Progress => Fix Committed
>
> ** Tags added: verification-needed verification-needed-cosmic
>
> ** Changed in: deja-dup (Ubuntu Bionic)
> Status: In Progress => Fix Committed
>
> ** Tags added: verification-needed-bionic
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1804744
>
> Title:
> Restoring from new location does not find backup files
>
> Status in Déjà Dup:
> Fix Released
> Status in deja-dup package in Ubuntu:
> Fix Released
> Status in deja-dup source package in Bionic:
> Fix Committed
> Status in deja-dup source package in Cosmic:
> Fix Committed
>
> Bug description:
> [Impact]
>
> When a user tries to restore a backup on a fresh Ubuntu install, deja-
> dup will not be able to find their backup at first.
>
> A workaround is to set the current backup location on the fresh
> install to the old backup location. But that's not at all an obvious
> path. A user would not normally try that.
>
> [Test Case]
>
> 1. Back up some files to a temporary dir.
> 2. gsettings reset-recursively org.gnome.DejaDup
> 3. Try to restore from that temporary dir.
>
> Notice that deja-dup will complain about waiting for Google Drive to
> be set up (even though your backup drive has nothing to do with
> Google).
>
> [Regression Potential]
>
> The proposed patch mostly affects the restore dialog. Any regressions
> would likely be in that flow.
>
> [Original Report]
>
> 1. I back up my computer to my external hard drive
> 2. I reinstall my new distro
> 3. I install deja-dup and duplicity
> 4. I open deja-dup, and select restor...

[Michael Terry (mterry)]

I'm able to confirm on bionic. Don't have my cosmic install handy anymore.

[Launchpad Janitor (janitor)]

This bug was fixed in the package deja-dup - 37.1-2fakesync1ubuntu0.1

---------------
deja-dup (37.1-2fakesync1ubuntu0.1) bionic; urgency=medium

  * debian/patches/lp-1804744.patch:
    - Fix bug preventing restore on a fresh install if you don't
      also set your current backup location to the old backup.
      LP: #1804744

 -- Michael Terry <email address hidden> Fri, 11 Jan 2019 21:49:35 -0500

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for deja-dup has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Sebastien Bacher (seb128)]

Cosmic indeed fails to restore the backup with the described steps, it does work after installing 38.0-1ubuntu0.1 from proposed, marking cosmic verified

[Launchpad Janitor (janitor)]

This bug was fixed in the package deja-dup - 38.0-1ubuntu0.1

---------------
deja-dup (38.0-1ubuntu0.1) cosmic; urgency=medium

  * debian/patches/lp-1804744.patch:
    - Fix bug preventing restore on a fresh install if you don't
      also set your current backup location to the old backup.
      LP: #1804744

 -- Michael Terry <email address hidden> Fri, 11 Jan 2019 21:33:04 -0500