rfc822 parser and intltools differ in extraction of translatable text

[Zygmunt Krynicki]

I just noticed that our multi-line translation story is entirely, entirely broken.

Here's what a new manage.py l10n command says about the manifest provider:

(venv.fx)zyga@fx:~/checkbox.here/plainbox/plainbox/impl/providers/manifest$ ./manage.py l10n
Note: native, raw, localized
In unit: <JobDefinition id:'2013.com.canonical.plainbox::collect-manifest' plugin:'user-interact'>
Internationalized field: 'description'
  'This job interactively asks the user about each manifest entry and stores the\nresult. This job can be omitted but the manifest may be incomplete unless it\nwas cached on an earlier run or provided externally.'
  'This job interactively asks the user about each manifest entry and stores the\nresult. This job can be omitted but the manifest may be incomplete unless it\nwas cached on an earlier run or provided externally.'
  'This job interactively asks the user about each manifest entry and stores the\nresult. This job can be omitted but the manifest may be incomplete unless it\nwas cached on an earlier run or provided externally.'
Internationalized field: 'summary'
  'Collect the hardware manifest (interactively)'
  'Collect the hardware manifest (interactively)'
  'Zbierz informacje dla manifest sprz\u0119tu (interaktywnie)'

(now you cannot obviously see the color coding I've copy-pasted from my terminal, the first line is native, second is raw and the last is localized. Looking at the paste above we can see that the "collect the hardware manifest" line is okay. It's translated into polish and indeed it works in practice during testing. Looking at the longer "This job interactively..." line we can see, however, that the line is not translated. That's odd, because this is what we get by de-compiling the .mo file:

(venv.fx)zyga@fx:~/checkbox.here/plainbox/plainbox/impl/providers/manifest$ msgunfmt build/mo/pl/LC_MESSAGES/plainbox-provider-manifest.mo

[snip]

msgid ""
"This job interactively asks the user about each manifest entry and stores "
"the result. This job can be omitted but the manifest may be incomplete "
"unless it was cached on an earlier run or provided externally."
msgstr ""
"To zadanie pyta użytkownika o dane potrzebne do skompletowania manifestu "
"sprzętu. To zadanie może zostać pominięte lecz w takim przypadku manifest "
"może być niekompletny, chyba, że jest on zapamiętany z poprzedniego "
"przebiegu testów lub został dostarczony z inny sposób."

[snip]

One thing that is blatantly obvious here is that the whole text doesn't have any newlines. Our parser currently preserves those newlines (and we might rely on that somewhere, I haven't checked yet). I suspect at least, that shell parsing might rely on it.

(venv.fx)zyga@fx:~/checkbox.here/plainbox/plainbox/impl/providers/manifest$ cat units/manifest.pxu
unit: job
id: collect-manifest
_summary: Collect the hardware manifest (interactively)
_description:
 This job interactively asks the user about each manifest entry and stores the
 result. This job can be omitted but the manifest may be incomplete unless it
 was cached on an earlier run or provided externally.
plugin: user-interact
command: plainbox-manifest-collect
estimated_duration: 30
flags: preserve-locale

With a small patch, which breaks a lot of carefully crafted tests, we get this behavior instead:

(venv.fx)zyga@fx:~/checkbox.here/plainbox/plainbox/impl/providers/manifest$ ./manage.py l10n
Note: native, raw, localized
In unit: <JobDefinition id:'2013.com.canonical.plainbox::collect-manifest' plugin:'user-interact'>
Internationalized field: 'description'
  'This job interactively asks the user about each manifest entry and stores the result. This job can be omitted but the manifest may be incomplete unless it was cached on an earlier run or provided externally.'
  'This job interactively asks the user about each manifest entry and stores the result. This job can be omitted but the manifest may be incomplete unless it was cached on an earlier run or provided externally.'
  'To zadanie pyta u\u017cytkownika o dane potrzebne do skompletowania manifestu sprz\u0119tu. To zadanie mo\u017ce zosta\u0107\xa0pomini\u0119te lecz w takim przypadku manifest mo\u017ce by\u0107\xa0niekompletny, chyba, \u017ce jest on zapami\u0119tany z poprzedniego przebiegu test\xf3w lub zosta\u0142 dostarczony z inny spos\xf3b.'
Internationalized field: 'summary'
  'Collect the hardware manifest (interactively)'
  'Collect the hardware manifest (interactively)'
  'Zbierz informacje dla manifest sprz\u0119tu (interaktywnie)'

Now everything is localized but we need to check how to properly get the translation input.

[Zygmunt Krynicki (zyga)]

Here's the patch (though it breaks tests) for reference