Bad parsing of GnuPG output, expecting english output, which doesn't work if locales different than english.

[az]

this is a forward/extract of debian bug #565398, which lives over there: http://bugs.debian.org/565398

summary: when using gpg and --encrypt-key, and only if LANG is set to de_AT
and LC_CTYPE is set to de_AT (or POSIX or C, but not iso-8859-1 or utf-8),
then incremental backups fail completely. somehow or other duplicity
mis-interprets gnupg output and errors out: i assume it would be
the 8-bit nature of certain phrases like "Schlüssel" (german for "key")
which causes the mess, but i can't identify the code in question.

reproducible: always. needs only an empty backup dir (source), an empty
target dir (target), a gpg key and locale de_AT available.

duplicity --encrypt-key KEYID source file://target
 works fine, creates a (minuscule) full backup
 running it again, with LANG unset (or to something purely 7bit like en_US)
 works perfect again, no problems.

but running it with LANG=de_AT and LC_CTYPE=de_AT causes duplicity to attempt
some gnupg decryption (of what? the cache and archive are in sync), and
it dies with a gpg failure (with the gpg message being the german equivalent
of "encryption failed: bad passphrase", not surprising as no passphrase was
given, nor should one be necessary).

i'm attaching the relevant -v9 logs, first of the successful full run,
then of a failed inc run (LANG and LC_CTYPE=de_AT) and then of the
successful inc run (LANG="", LC_CTYPE=de_AT). no changes were made to
the source dir between runs.

regards,
az

[Richard (rimcaw)]

Similar problem here (failure if LANG=fr_FR.UTF-8, success if LANG is empty)

[Peter Wu (lekensteyn)]

LANG= does not work for me, neither did LC_CTYPE= or setting these environment variables to C.

LC_MESSAGES=C does work.
Output of `locale` (after exporting LC_MESSAGES):
LANG=nl_NL.UTF-8
LANGUAGE=nl_NL
LC_CTYPE="nl_NL.UTF-8"
LC_NUMERIC="nl_NL.UTF-8"
LC_TIME="nl_NL.UTF-8"
LC_COLLATE="nl_NL.UTF-8"
LC_MONETARY="nl_NL.UTF-8"
LC_MESSAGES=C
LC_PAPER="nl_NL.UTF-8"
LC_NAME="nl_NL.UTF-8"
LC_ADDRESS="nl_NL.UTF-8"
LC_TELEPHONE="nl_NL.UTF-8"
LC_MEASUREMENT="nl_NL.UTF-8"
LC_IDENTIFICATION="nl_NL.UTF-8"
LC_ALL=

Setting LC_ALL=C works too, but changes other locale information too (unknown effects)

[Kenneth Loafman (kenneth-loafman)]

As a workaround, how about doing this:
  LC_ALL=C duplicity full ...
This will set the locale for duplicity only and not affect other programs.

I know what the problem is and am having fits trying to find the
documentation for GnuPG error codes and what they produce as an error
message. Currently duplicity uses the text of the error to determine what
happened. It should respond to the error code, which thankfully, does not
change with the locale.

...Ken

On Sat, Jun 11, 2011 at 11:51 AM, Lekensteyn <email address hidden>wrote:

> LANG= does not work for me, neither did LC_CTYPE= or setting these
> environment variables to C.
>
> LC_MESSAGES=C does work.
> Output of `locale` (after exporting LC_MESSAGES):
> LANG=nl_NL.UTF-8
> LANGUAGE=nl_NL
> LC_CTYPE="nl_NL.UTF-8"
> LC_NUMERIC="nl_NL.UTF-8"
> LC_TIME="nl_NL.UTF-8"
> LC_COLLATE="nl_NL.UTF-8"
> LC_MONETARY="nl_NL.UTF-8"
> LC_MESSAGES=C
> LC_PAPER="nl_NL.UTF-8"
> LC_NAME="nl_NL.UTF-8"
> LC_ADDRESS="nl_NL.UTF-8"
> LC_TELEPHONE="nl_NL.UTF-8"
> LC_MEASUREMENT="nl_NL.UTF-8"
> LC_IDENTIFICATION="nl_NL.UTF-8"
> LC_ALL=
>
> Setting LC_ALL=C works too, but changes other locale information too
> (unknown effects)
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/510625
>
> Title:
> 0.6.06: all incremental backends with --encrypt-key and LANG=de_AT
>
> Status in Duplicity - Bandwidth Efficient Encrypted Backup:
> New
> Status in Debian GNU/Linux:
> Confirmed
>
> Bug description:
> this is a forward/extract of debian bug #565398, which lives over
> there: http://bugs.debian.org/565398
>
> summary: when using gpg and --encrypt-key, and only if LANG is set to
> de_AT
> and LC_CTYPE is set to de_AT (or POSIX or C, but not iso-8859-1 or utf-8),
> then incremental backups fail completely. somehow or other duplicity
> mis-interprets gnupg output and errors out: i assume it would be
> the 8-bit nature of certain phrases like "Schlüssel" (german for "key")
> which causes the mess, but i can't identify the code in question.
>
> reproducible: always. needs only an empty backup dir (source), an empty
> target dir (target), a gpg key and locale de_AT available.
>
> duplicity --encrypt-key KEYID source file://target
> works fine, creates a (minuscule) full backup
> running it again, with LANG unset (or to something purely 7bit like
> en_US)
> works perfect again, no problems.
>
> but running it with LANG=de_AT and LC_CTYPE=de_AT causes duplicity to
> attempt
> some gnupg decryption (of what? the cache and archive are in sync), and
> it dies with a gpg failure (with the gpg message being the german
> equivalent
> of "encryption failed: bad passphrase", not surprising as no passphrase
> was
> given, nor should one be necessary).
>
> i'm attaching the relevant -v9 logs, first of the successful full run,
> then of a failed inc run (LANG and LC_CTYPE=de_AT) and then of the
> successful inc run (LANG="", LC_CTYPE=de_AT). no changes were made to
> the source dir between runs.
>
> regards,
> az
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/510625/+subscriptions
>

[Nuno Sucena Almeida (slug-debian)]

I'm having a similar issue with ubuntu 11.10, but with another locale. As a workaround on my backup (bash) script, which calls duplicity, I have at the top:

#!/usr/bin/env bash

LANG=""
LANGUAGE=""

(...)

duplicity ...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Zertrin (zertrin)]

The bug occurs there :

http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/collections.py#L222

Duplicity is parsing the output of GPG and expecting english output...

[Zertrin (zertrin)]

Bug located in the source code.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[az (az-debian)]

hmmm. gpg does report its status and needs nicely on the status fd, and in a fixed format (NEED_PASSPHRASE...)
so it looks like the gpg.py module needs to perform a bit more status_fd handling and reporting. right now it only extracts signature key ids.