Duplicity/deja-dup will not create a backup

[John Bell]

I get the following message:

Error processing remote manifest (duplicity-inc.20190214T141809Z.to.20190221T175834Z.manifest.gpg): GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: WARNING: "--no-use-agent" is an obsolete option - it has no effect
gpg: AES256 encrypted data
gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key
===== End GnuPG log =====

Sometimes I get an alternate error message:

Error creating directory /media/john/8126ba33-ddc9-401d-8ee0-0b59a0bada53: Permission denied

(address of my internal disk is correct)

Problem started last week when I tried to deal with multiple keyring requests that I tried to cull, but must have gotten rid of one too many. Or it is something different.

I am on Ubuntu 18.04.2 LTS

deja-dup 37.1-2fakesync1ubuntu0.1
duplicity 0.7.17-0ubuntu1.1

org.gnome.DejaDup last-restore '2019-02-05T06:46:10.011815Z'
org.gnome.DejaDup periodic true
org.gnome.DejaDup periodic-period 7
org.gnome.DejaDup full-backup-period 90
org.gnome.DejaDup backend 'local'

Error processing remote manifest (duplicity-inc.20190214T141809Z.to.20190221T175834Z.manifest.gpg): GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: WARNING: "--no-use-agent" is an obsolete option - it has no effect
gpg: AES256 encrypted data
gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key
===== End GnuPG log =====
org.gnome.DejaDup last-run '2019-02-21T18:17:28.744944Z'
org.gnome.DejaDup nag-check '2019-02-07T14:18:30.009530Z'
org.gnome.DejaDup prompt-check '2019-02-05T02:56:12.635538Z'
org.gnome.DejaDup root-prompt true
org.gnome.DejaDup include-list ['$HOME']
org.gnome.DejaDup exclude-list ['$TRASH', '$DOWNLOAD']
org.gnome.DejaDup last-backup '2019-02-21T18:17:28.744944Z'
org.gnome.DejaDup allow-metered false
org.gnome.DejaDup delete-after 0
org.gnome.DejaDup.Rackspace username ''
org.gnome.DejaDup.Rackspace container 'john-ThinkCentre-M910t'
org.gnome.DejaDup.S3 id ''
org.gnome.DejaDup.S3 bucket ''
org.gnome.DejaDup.S3 folder 'john-ThinkCentre-M910t'
org.gnome.DejaDup.OpenStack authurl ''
org.gnome.DejaDup.OpenStack tenant ''
org.gnome.DejaDup.OpenStack username ''
org.gnome.DejaDup.OpenStack container 'john-ThinkCentre-M910t'
org.gnome.DejaDup.GCS id ''
org.gnome.DejaDup.GCS bucket ''
org.gnome.DejaDup.GCS folder 'john-ThinkCentre-M910t'
org.gnome.DejaDup.Local folder '/media/john/8126ba33-ddc9-401d-8ee0-0b59a0bada53/Backups'
org.gnome.DejaDup.Remote uri ''
org.gnome.DejaDup.Remote folder 'john-ThinkCentre-M910t'
org.gnome.DejaDup.Drive uuid ''
org.gnome.DejaDup.Drive icon ''
org.gnome.DejaDup.Drive folder '$HOSTNAME'
org.gnome.DejaDup.Drive name ''
org.gnome.DejaDup.GOA id ''
org.gnome.DejaDup.GOA folder 'john-ThinkCentre-M910t'
org.gnome.DejaDup.GOA type 'google'
org.gnome.DejaDup.File short-name ''
org.gnome.DejaDup.File type 'normal'
org.gnome.DejaDup.File migrated true
org.gnome.DejaDup.File name ''
org.gnome.DejaDup.File path ''
org.gnome.DejaDup.File uuid ''
org.gnome.DejaDup.File icon ''
org.gnome.DejaDup.File relpath @ay []

[Michael Terry (mterry)]

Hello! Sorry for slow response.

1) the error message from duplicity indicates a bad encryption password.

2) The settings you posited are confusing. The settings indicate that you are backing up to a local folder. But your error messages indicate that you’re backing up to a pluggable drrive. Which is the reality?

[John Bell (jkabell)]

Thanks Michael,

I am backing up to a separate 1 TB SATA drive in my desktop tower
directly plugged to the motherboard.  I think that makes it a local
folder?  This drive has a folder 'Backups' originally created by
Deja-Dup/Duplicity when under Ubuntu 16.04 and I select that as the
location for my backups.  If I just call out the drive and not the
folder Deja-Dup/Duplicity will try to make a full backup on the drive
outside that folder and runs out of room as it ignores the previous backups.

The rest of this may be over-kill, but for completeness this is what I have:

The drive being backed up where the OS and Home folder reside is a
separate SATA

This computer also has a solid state 128 GB drive with Windows which I
do not back up.

History:  I upgraded from Ubuntu 16.04 to 18.04 some months ago and ran
into a load of problems mainly from programs that no longer ran but I
managed to resurrect over time.  Wine became, and remains, a real
problem and some of my old Windows programs became unusable including a
PaperPort/Nuance program essential to read some old ScanSoft data which
is serious to me because I neglected to convert their oddball files to
pdf while I could under Ubuntu 16.04.  But that is not relevant. *I
ended up backing up and reinstalling Ubuntu 18.04 completely ending up
with odd partitioning which I think is limiting my ability to handle
large files - a separate issue I need help on maybe as a separate
submission.*

Thanks again, Michael.

John Bell

On 3/4/19 10:04 PM, Michael Terry wrote:

> Hello! Sorry for slow response.
>
> 1) the error message from duplicity indicates a bad encryption password.
>
> 2) The settings you posited are confusing. The settings indicate that
> you are backing up to a local folder. But your error messages indicate
> that you’re backing up to a pluggable drrive. Which is the reality?
>

[John Bell (jkabell)]

Michael,

How do I correct a bad encryption password?  There are only two
possibilities I would have used but I don't see where I could enter it.
I was asked for a keyring once when starting my backup and maybe I used
the wrong one then, but it has never asked me again.

Is there something wrong in my selection of 'local folder' for a
separate SATA drive on my computer?  I am sure that is the way I have
always designated it.

It will be coming up for two weeks without being able to back up, any
help would be appreciated.

John Bell

On 3/4/19 10:04 PM, Michael Terry wrote:
> Hello! Sorry for slow response.
>
> 1) the error message from duplicity indicates a bad encryption password.
>
> 2) The settings you posited are confusing. The settings indicate that
> you are backing up to a local folder. But your error messages indicate
> that you’re backing up to a pluggable drrive. Which is the reality?
>

[Michael Terry (mterry)]

Hello, sorry for slow response.

Selecting 'local folder' should be fine. That seems appropriate for your case.

The permission error you're getting is odd. Is the drive sometimes mounted differently?

To clear out any saved password, open the "Passwords and Keys" app and look for the saved "Backup encryption password". Delete that and re-run your backup. It should prompt for the password.

If for some reason, that still doesn't work, the worst case scenario is to back up to a new folder. This will create a new backup series and you can set a new password.

(I'm sorry you're having these problems! Let me know if any of these attempts help.)

[John Bell (jkabell)]

Michael,

Thanks for still helping me with this.

Just to update you on where I am:  As it had been a month without a
backup, I gave in and bought another 1TB internal SATA drive and
formatted it as Ext4.  The first backup on this drive created a bunch of
files but then stopped with a Backup Failed message.  I checked for
advice on the internet and someone had been successful by formatting the
disk as NTFS, so I deleted the whole new drive and reformatted - a long
process.

I then ran a backup with no encryption password and it succeeded,
creating a full backup.

Today Deja Dup attempted a backup on its original Thursday schedule
(last backup was 5 days ago and the schedule is weekly so it must be
back to its original setup)  and failed leaving me with the following
message:

     Error creating directory /media/john/20190323 Backups: Permission
denied

I have just followed your instruction to delete the backup encryption
password and it has just given me the same error message, not prompting
me for a password.  I will reboot and try again.  If that doesn't work I
will have to delete my last backup to create enough room on the disk
(full backup is a little under 600GB so only 400GB left) for another
full backup, which leaves me a little vulnerable, and give a new folder
name.

Rebooting was no help.

Why is it trying to create 'directory /media/john/20190323 Backups',
have I specified the location incorrectly?  The new drive is called
'20290323 Backups' and the last full backup created no separate folder
in that drive.  The first entry in that disk is
'duplicity-full.20190323T183857Z.manifest' and the rest of the entries
are separate folders called xxxx.volNNN.difftar.gz each 52.4 MB.

I also get frequent, almost daily, Ubuntu crash reports which I seldom
got under Ubuntu 16.04.  I attach screenshots covering the whole content
of the error with all the topics compressed, but I do have separate
screenshots with full details if you think any would help.  Zeitgeist
was the prime cause of the problem on this occasion.

Thank you very much for your help.

John Bell

On 3/23/19 3:16 PM, Michael Terry wrote:
> Hello, sorry for slow response.
>
> Selecting 'local folder' should be fine. That seems appropriate for your
> case.
>
> The permission error you're getting is odd. Is the drive sometimes
> mounted differently?
>
> To clear out any saved password, open the "Passwords and Keys" app and
> look for the saved "Backup encryption password". Delete that and re-run
> your backup. It should prompt for the password.
>
> If for some reason, that still doesn't work, the worst case scenario is
> to back up to a new folder. This will create a new backup series and you
> can set a new password.
>
> (I'm sorry you're having these problems! Let me know if any of these
> attempts help.)
>

[John Bell (jkabell)]

UPDATE - just got a notification that I had a successful backup - second
in 5 days.  This time it created another Manifest file and two folders -
a difftar.gz and an sigtar.gz.  Happy with that but it is weird that I
got the errors creating directories first.  I will see what happened
next Thursday when it is next scheduled.  I thought I got an error from
the automatic backup and then again when I tried it manually so I don't
know why it tried again but I am happy with the outcome.  It does seem
to be a bug.

John

On 3/28/19 9:54 AM, John Bell wrote:
> Michael,
>
> Thanks for still helping me with this.
>
> Just to update you on where I am:  As it had been a month without a
> backup, I gave in and bought another 1TB internal SATA drive and
> formatted it as Ext4.  The first backup on this drive created a bunch
> of files but then stopped with a Backup Failed message.  I checked for
> advice on the internet and someone had been successful by formatting
> the disk as NTFS, so I deleted the whole new drive and reformatted - a
> long process.
>
> I then ran a backup with no encryption password and it succeeded,
> creating a full backup.
>
> Today Deja Dup attempted a backup on its original Thursday schedule
> (last backup was 5 days ago and the schedule is weekly so it must be
> back to its original setup)  and failed leaving me with the following
> message:
>
>     Error creating directory /media/john/20190323 Backups: Permission
> denied
>
> I have just followed your instruction to delete the backup encryption
> password and it has just given me the same error message, not
> prompting me for a password.  I will reboot and try again.  If that
> doesn't work I will have to delete my last backup to create enough
> room on the disk (full backup is a little under 600GB so only 400GB
> left) for another full backup, which leaves me a little vulnerable,
> and give a new folder name.
>
> Rebooting was no help.
>
> Why is it trying to create 'directory /media/john/20190323 Backups',
> have I specified the location incorrectly?  The new drive is called
> '20290323 Backups' and the last full backup created no separate folder
> in that drive.  The first entry in that disk is
> 'duplicity-full.20190323T183857Z.manifest' and the rest of the entries
> are separate folders called xxxx.volNNN.difftar.gz each 52.4 MB.
>
> I also get frequent, almost daily, Ubuntu crash reports which I seldom
> got under Ubuntu 16.04.  I attach screenshots covering the whole
> content of the error with all the topics compressed, but I do have
> separate screenshots with full details if you think any would help.
> Zeitgeist was the prime cause of the problem on this occasion.
>
> Thank you very much for your help.
>
> John Bell
>
> On 3/23/19 3:16 PM, Michael Terry wrote:
>> Hello, sorry for slow response.
>>
>> Selecting 'local folder' should be fine. That seems appropriate for your
>> case.
>>
>> The permission error you're getting is odd. Is the drive sometimes
>> mounted differently?
>>
>> To clear out any saved password, open the "Passwords and Keys" app and
>> look for the saved "Backup encryption password". Delete that and re-run
>> your b...

[Michael Terry (mterry)]

I know there were a few issues in this report. The first you mentioned about a GnuPG error is likely due to a recently fixed issue around passphrases:

https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019

The other issues are not as clear to me. But you say you aren’t hitting them now. I will optimistically close this report.

[John Bell (jkabell)]

Michael,

I suppose the main issue has been resolved in that I put in a new hard
drive and have been able to MANUALLY back up each week so I am happy
with that.  However, each week the system tries to do the scheduled
backup and runs into the same problem of inability to create a directory
in that same drive.  Somewhere the is a file for Deja-Dup with incorrect
data (password?) for the scheduled backup.  How that happened I do not
know although it is possible I confused passwords at some point.  But it
should be able to pick up again using the password I provide when I do
the manual backup.  Does that constitute a bug?

John Bell

On 4/7/19 12:17 PM, Michael Terry wrote:
> I know there were a few issues in this report. The first you mentioned
> about a GnuPG error is likely due to a recently fixed issue around
> passphrases:
>
> https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019
>
> The other issues are not as clear to me. But you say you aren’t hitting
> them now. I will optimistically close this report.
>
> ** Changed in: deja-dup
> Status: New => Fix Released
>

[John Bell (jkabell)]

Michael,

I replied before I had clicked on your link.  The Confirmation showed I
had a bad password somewhere, but it kept repeating when I tried an
alternate possible password spelling I may have used so I am not so sure
that gave me a clear guide.

I tried to discover the bad password, but Passwords and Keys did not
have an entry for Backups so that did not work for me.

I have installed the beta Deja-Dup and have done a manual backup with it
already which it did without asking me for a password or key.  The
scheduled backup is Thursday so I live in hopes that it will happen
automatically.

Thank you.

John Bell

On 4/8/19 2:16 PM, John Bell wrote:
> Michael,
>
> I suppose the main issue has been resolved in that I put in a new hard
> drive and have been able to MANUALLY back up each week so I am happy
> with that.  However, each week the system tries to do the scheduled
> backup and runs into the same problem of inability to create a
> directory in that same drive.  Somewhere the is a file for Deja-Dup
> with incorrect data (password?) for the scheduled backup.  How that
> happened I do not know although it is possible I confused passwords at
> some point.  But it should be able to pick up again using the password
> I provide when I do the manual backup.  Does that constitute a bug?
>
> John Bell
>
> On 4/7/19 12:17 PM, Michael Terry wrote:
>> I know there were a few issues in this report. The first you mentioned
>> about a GnuPG error is likely due to a recently fixed issue around
>> passphrases:
>>
>> https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019
>>
>> The other issues are not as clear to me. But you say you aren’t hitting
>> them now. I will optimistically close this report.
>>
>> ** Changed in: deja-dup
>>         Status: New => Fix Released
>>

[John Bell (jkabell)]

Michael,

The problem returned two weeks ago and I am now 15 days without a
backup.?? I think a software update undid the fix.?? I updated deja-dup by
running

snap install deja-dup --classic

but the problem continues.?? Did a new version of deja-dup undo the fix???
Should I file a new bug?

John Bell

On 4/7/19 12:17 PM, Michael Terry wrote:
> I know there were a few issues in this report. The first you mentioned
> about a GnuPG error is likely due to a recently fixed issue around
> passphrases:
>
> https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019
>
> The other issues are not as clear to me. But you say you aren???t hitting
> them now. I will optimistically close this report.
>
> ** Changed in: deja-dup
> Status: New => Fix Released
>

[John Bell (jkabell)]

Michael,

Just to add to my note:?? I see a lot of complaints that Restore doesn't
work anymore with Deja-Dup.?? Can you please confirm that I will be able
to Restore if needed (I have done full Restores only about five times in
my life, but they are necessary).?? If I won't be able to restore, what
is the point of backing up?

John Bell

On 5/4/19 7:10 PM, John Bell wrote:
>
> Michael,
>
> The problem returned two weeks ago and I am now 15 days without a
> backup.?? I think a software update undid the fix.?? I updated deja-dup
> by running
>
> snap install deja-dup --classic
> but the problem continues.?? Did a new version of deja-dup undo the
> fix??? Should I file a new bug?
>
> John Bell
>
>
> On 4/7/19 12:17 PM, Michael Terry wrote:
>> I know there were a few issues in this report. The first you mentioned
>> about a GnuPG error is likely due to a recently fixed issue around
>> passphrases:
>>
>> https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019
>>
>> The other issues are not as clear to me. But you say you aren???t hitting
>> them now. I will optimistically close this report.
>>
>> ** Changed in: deja-dup
>> Status: New => Fix Released
>>

[John Bell (jkabell)]

Michael,

Sorry to trouble you with so many emails.?? I was checking if I still had
enough space on my separate internal disk when I noticed it was not
mounted.?? Mounted it, found enough space, and the backup has worked with
no problem.?? Sorry to trouble you.?? That bug can remain closed.?? Now if
I could only find why that disk does not mount automatically ...

John Bell

On 5/4/19 7:17 PM, John Bell wrote:
>
> Michael,
>
> Just to add to my note:?? I see a lot of complaints that Restore
> doesn't work anymore with Deja-Dup.?? Can you please confirm that I
> will be able to Restore if needed (I have done full Restores only
> about five times in my life, but they are necessary).?? If I won't be
> able to restore, what is the point of backing up?
>
> John Bell
>
> On 5/4/19 7:10 PM, John Bell wrote:
>>
>> Michael,
>>
>> The problem returned two weeks ago and I am now 15 days without a
>> backup.?? I think a software update undid the fix.?? I updated deja-dup
>> by running
>>
>> snap install deja-dup --classic
>> but the problem continues.?? Did a new version of deja-dup undo the
>> fix??? Should I file a new bug?
>>
>> John Bell
>>
>>
>> On 4/7/19 12:17 PM, Michael Terry wrote:
>>> I know there were a few issues in this report. The first you mentioned
>>> about a GnuPG error is likely due to a recently fixed issue around
>>> passphrases:
>>>
>>> https://wiki.gnome.org/Apps/DejaDup/PassphraseProblems2019
>>>
>>> The other issues are not as clear to me. But you say you aren???t hitting
>>> them now. I will optimistically close this report.
>>>
>>> ** Changed in: deja-dup
>>> Status: New => Fix Released
>>>