rosetta-approve-imports.py is crashing

[Steve McInerney]

Seen several times today:

2010-01-06 07:31:52 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:52 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:52 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:56 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:56 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:59 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:59 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:31:59 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:00 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:00 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:00 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:00 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:01 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:01 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:03 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:14 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:16 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
2010-01-06 07:32:17 WARNING Found 2 competing templates with translation domain 'kabc_dir': "kabc-dir in Ubuntu Lucid package "kdelibs""; "kabc-dir in Ubuntu Lucid package "kdepimlibs"".
Traceback (most recent call last):
  File "/srv/launchpad.net/production/launchpad/cronscripts/rosetta-approve-imports.py", line 19, in <module>
    script.lock_and_run()
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/services/scripts/base.py", line 290, in lock_and_run
    implicit_begin=implicit_begin, isolation=isolation)
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/services/scripts/base.py", line 248, in run
    self.main()
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/scripts/import_queue_gardener.py", line 32, in main
    if translation_import_queue.executeOptimisticApprovals(self.txn):
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/translationimportqueue.py", line 1154, in executeOptimisticApprovals
    guess = entry.getGuessedPOFile()
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/translationimportqueue.py", line 489, in getGuessedPOFile
    pofile = self._guess_multiple_directories_with_pofile()
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/translationimportqueue.py", line 682, in _guess_multiple_directories_with_pofile
    lang_code, translation_domain)
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/translationimportqueue.py", line 454, in _get_pofile_from_language
    pofile.setPathIfUnique(self.path)
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/pofile.py", line 1011, in setPathIfUnique
    if path != self.path and self.potemplate.isPOFilePathAvailable(path):
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/potemplate.py", line 689, in isPOFilePathAvailable
    self.sourcepackagename)
  File "/srv/launchpad.net/production/launchpad-rev-8806/lib/lp/translations/model/pofile.py", line 1572, in getPOFileByPathAndOrigin
    clauseTables=['POTemplate'])
  File "/home/pqm/for_rollouts/production/eggs/storm-0.15danilo_storm_launchpad_r342-py2.5-linux-x86_64.egg/storm/sqlobject.py", line 352, in selectOne
  File "/home/pqm/for_rollouts/production/eggs/storm-0.15danilo_storm_launchpad_r342-py2.5-linux-x86_64.egg/storm/sqlobject.py", line 516, in _one
  File "/home/pqm/for_rollouts/production/eggs/storm-0.15danilo_storm_launchpad_r342-py2.5-linux-x86_64.egg/storm/store.py", line 1117, in one
storm.exceptions.NotOneError: one() used with more than one result available

[Henning Eggers (henninge)]

I had Tom Haddon run this query to try to find the offending entry:

select
    po1.id as po1id, po2.id as po2id,
    pot1.id as pot1id, pot1.name as pot1name,
    pot2.id as pot2id, pot2.name as pot2name,
    po1.path, distroseries.name, sourcepackagename.name
from
    pofile po1, pofile po2, potemplate pot1, potemplate pot2,
    distroseries, sourcepackagename
where
    po1.id <> po2.id and
    po1.potemplate = pot1.id and
    po2.potemplate = pot2.id and
    po1.path = po2.path and
    pot1.distroseries = pot2.distroseries and
    pot1.sourcepackagename = pot2.sourcepackagename and
    pot1.distroseries = distroseries.id and
    pot1.sourcepackagename = sourcepackagename.id;

The output is attached, is has twice as many lines as needed because the "<>" up there should have been a "<", so please ignore every other line.

So, some of these come about through inactive templates but others don't. I am not sure what is wrong here, code or data integrity.

[Henning Eggers (henninge)]

Ok, danilo says this might be a lock-up between this script and the message sharing migration script that is currently running.

[Jeroen T. Vermeulen (jtv)]

Based on the place this happens in _guess_multiple_directories_with_pofile, this is one of the "cross-approved" KDE templates. The name of the source package that the offending files were uploaded to must begin with kde-i18n- or kde-l10n-.

[Jeroen T. Vermeulen (jtv)]

Updated query: https://pastebin.canonical.com/26351/
Result: https://pastebin.canonical.com/26353/

The problem was template kabc-net, in source package kdepimplibs, in Lucid. There's also a disabled template kabc-net-obsolete. Each of these has one POFile with path "messages/kdepimlibs/kabc_net.po":

In kabc-net (POTemplate 33083), POFile 1288528, language kk (Kazakh).
In kabc-net-obsolete (POTemplate 31513), POFile 1225200, language km (Khmer).

These uploads shouldn't have been approved with those names. Of course that's no justification for the approver to break though.

[Jeroen T. Vermeulen (jtv)]

The fix is a small "update" query for the obsolete POFile. Still trying to get someone to approve its execution.

[Jeroen T. Vermeulen (jtv)]

Looks like the SQL has been executed, and the approver is going again! Also attaching a branch that should prevent recurrence.

[Ursula Junque (ursinha)]

This bug was mentioned in stable r10152 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/10152>