upstream-translation-update Jenkins job failing

Bug #1299349 reported by Andreas Jaeger on 2014-03-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Medium
Andreas Jaeger
Cinder
Medium
Andreas Jaeger
Glance
Medium
Thierry Carrez
OpenStack Compute (nova)
Medium
Andreas Jaeger
OpenStack Heat
High
Andreas Jaeger
OpenStack Identity (keystone)
High
Andreas Jaeger
neutron
High
Andreas Jaeger
openstack-manuals
Medium
Andreas Jaeger

Bug Description

Various upstream-translation-update succeed apparently but looking into the file errors get ignored - and there are errors uploading the files to the translation site like:

Error uploading file: There is a syntax error in your file.
Line 1936: duplicate message definition...
Line 7: ...this is the location of the first definition

For keystone, see:
http://logs.openstack.org/78/7882359da114079e8411bd3f97c5628f2cd1c098/post/keystone-upstream-translation-update/27cbb22/

This has been fixed on ironic with:
https://review.openstack.org/#/c/83935
See also: http://lists.openstack.org/pipermail/openstack-i18n/2014-March/000479.html

Changed in keystone:
assignee: nobody → Andreas Jaeger (jaegerandi)
status: New → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/83951

Changed in cinder:
assignee: nobody → Andreas Jaeger (jaegerandi)
status: New → In Progress
Changed in ceilometer:
assignee: nobody → Andreas Jaeger (jaegerandi)
status: New → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/83960

Changed in heat:
assignee: nobody → Andreas Jaeger (jaegerandi)
status: New → In Progress
Andreas Jaeger (jaegerandi) wrote :
Changed in nova:
status: New → In Progress
assignee: nobody → Andreas Jaeger (jaegerandi)
Łukasz Jernaś (deejay1) wrote :

Hmm, IMHO that looks like a bug in the transifex client. Do we know which version is on the doc builders?

tags: added: i18n
Andreas Jaeger (jaegerandi) wrote :

AFAIK it's version 1.2.1 of the transifex client - we always use the latest release.

Reviewed: https://review.openstack.org/81925
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=e8d8306b21cb59d600c7500d6cdf2effecc3a679
Submitter: Jenkins
Branch: master

commit e8d8306b21cb59d600c7500d6cdf2effecc3a679
Author: Andreas Jaeger <email address hidden>
Date: Thu Mar 20 22:34:47 2014 +0100

    Fix Jenkins translation jobs

    The jobs keystone-propose-translation-update and
    keystone-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where obsolete
    entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Closes-Bug: #1299349

    Change-Id: Ibbf26bb8158735a5dc57bff7d77a49e00239857c

Changed in keystone:
status: In Progress → Fix Committed
Andreas Jaeger (jaegerandi) wrote :

As discussed on #openstack-infra and #openstack-cinder, I'm proposing patches for the pep8 queue that check the locale files with "msgfmt --check-format" so that no new errors can get introduced.

Changed in heat:
milestone: none → juno-1
importance: Undecided → High

Fix proposed to branch: master
Review: https://review.openstack.org/84234

Changed in neutron:
assignee: nobody → Andreas Jaeger (jaegerandi)
status: New → In Progress
Dolph Mathews (dolph) wrote :

Should this be backported to keystone's milestone-proposed for icehouse?

Andreas Jaeger (jaegerandi) wrote :

Dolph, yes I suggest to backport this since I assume to compilation of message catalogs will fail otherwise. Will you so?

Dolph Mathews (dolph) wrote :

Andreas: Happy to, although I just tried and patchset 3 of https://review.openstack.org/#/c/81925/ conflicts with milestone-proposed... I assume you automated the process of removing the duplicates - can you share that? Thanks!

Andreas Jaeger (jaegerandi) wrote :

Dolph, I used the script from comment #2 in:
https://bugs.launchpad.net/ironic/+bug/1298645

Just run it over all po files.

Reviewed: https://review.openstack.org/83960
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=8ec3bbb7cf0400b6b0ae82f88aad45e3fb366ab6
Submitter: Jenkins
Branch: master

commit 8ec3bbb7cf0400b6b0ae82f88aad45e3fb366ab6
Author: Andreas Jaeger <email address hidden>
Date: Sat Mar 29 09:38:30 2014 +0100

    Fix Jenkins translation jobs

    The jobs heat-propose-translation-update and
    heat-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Change-Id: I2071e3a792d107840449bbfe39e3a540ea1b626f
    Closes-Bug: #1299349

Changed in heat:
status: In Progress → Fix Committed
Dolph Mathews (dolph) on 2014-04-01
Changed in keystone:
importance: Undecided → High
Thierry Carrez (ttx) on 2014-04-01
tags: added: icehouse-rc-potential
Dolph Mathews (dolph) wrote :

Thanks Andreas, I added a bit more automation to that script so you can just point it at the entire locale/ directory:

  python clean_po.py keystone/locale/

See: https://gist.github.com/dolph/9915293

Reviewed: https://review.openstack.org/83951
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c72cc4b2345e9e271205d1de00c9812d0a2684ec
Submitter: Jenkins
Branch: master

commit c72cc4b2345e9e271205d1de00c9812d0a2684ec
Author: Andreas Jaeger <email address hidden>
Date: Sat Mar 29 06:38:52 2014 +0100

    Fix Jenkins translation jobs

    The jobs cinder-propose-translation-update and
    cinder-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Change-Id: I6d41dbdcc41646fcbd1ee84ce48cb0c461cd454c
    Closes-Bug: #1299349

Changed in cinder:
status: In Progress → Fix Committed
Changed in neutron:
importance: Undecided → High

Reviewed: https://review.openstack.org/84241
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=9e9dc87d3feef7ed07d8416004d4a12839c3dbc7
Submitter: Jenkins
Branch: master

commit 9e9dc87d3feef7ed07d8416004d4a12839c3dbc7
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 22:39:42 2014 +0200

    Check that all po/pot files are valid

    Perform minimal checks on po and pot files so that no broken files can
    get imported.

    Change-Id: Ic39d68106259a225fa5e5399123063e7c35ba150
    Co-Authored-By: Clark Boylan <email address hidden>
    Related-Bug: #1299349

Thierry Carrez (ttx) on 2014-04-02
no longer affects: keystone/icehouse
Changed in keystone:
milestone: none → icehouse-rc2

Reviewed: https://review.openstack.org/84243
Committed: https://git.openstack.org/cgit/openstack/operations-guide/commit/?id=6c24c72de4f2cacf298985a59bbbb6f7e6adb629
Submitter: Jenkins
Branch: master

commit 6c24c72de4f2cacf298985a59bbbb6f7e6adb629
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 22:45:07 2014 +0200

    Check that all po/pot files are valid

    Perform minimal checks on po and pot files so that no broken files can
    get imported.

    Change-Id: I9b5edbad9f4302ca24e4255f503ee2db45ebd5d7
    Co-Authored-By: Clark Boylan <email address hidden>
    Related-Bug: #1299349

Thierry Carrez (ttx) on 2014-04-03
Changed in keystone:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-03
Changed in nova:
milestone: none → icehouse-rc2
importance: Undecided → Medium
Thierry Carrez (ttx) on 2014-04-03
Changed in cinder:
milestone: none → icehouse-rc2
importance: Undecided → Medium

Reviewed: https://review.openstack.org/84563
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=c773cf96a52924a7dc42f774c57463e8ed66c9ef
Submitter: Jenkins
Branch: milestone-proposed

commit c773cf96a52924a7dc42f774c57463e8ed66c9ef
Author: Andreas Jaeger <email address hidden>
Date: Sat Mar 29 06:38:52 2014 +0100

    Fix Jenkins translation jobs

    The jobs cinder-propose-translation-update and
    cinder-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Change-Id: I6d41dbdcc41646fcbd1ee84ce48cb0c461cd454c
    Closes-Bug: #1299349
    (cherry picked from commit c72cc4b2345e9e271205d1de00c9812d0a2684ec)

Changed in cinder:
status: Fix Committed → Fix Released

Reviewed: https://review.openstack.org/84234
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=085a35d657cf0fa41a402f2af66c4beaa0f60db2
Submitter: Jenkins
Branch: master

commit 085a35d657cf0fa41a402f2af66c4beaa0f60db2
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 22:17:18 2014 +0200

    Fix Jenkins translation jobs

    The job neutron-propose-translation-update
    does not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the job.

    Add test to pep8 that checks that no new breakages get
    introduced.

    Change-Id: I9c680eeefecfa124c5b2900b7fa7a0a8f7e8ebd6
    Closes-Bug: #1299349

Changed in neutron:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2014-04-03
Changed in heat:
milestone: juno-1 → icehouse-rc2
Thierry Carrez (ttx) on 2014-04-04
Changed in neutron:
milestone: none → icehouse-rc2
John Griffith (john-griffith) wrote :

Just a note for other projects... Did a comparison of Dolph's script versus what I did in Cinder manually and we achieved the same results.

HIGHLY recommend Dolph's script here and just to highlight it a bit:
https://bugs.launchpad.net/keystone/+bug/1299349/comments/25

Dolph Mathews (dolph) wrote :

Advertised the fix for this (especially for milestone-proposed branches) on openstack-dev:

  http://lists.openstack.org/pipermail/openstack-dev/2014-April/031918.html

Reviewed: https://review.openstack.org/84233
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=977766080993c22ed279889c904bcc19883ff8b9
Submitter: Jenkins
Branch: master

commit 977766080993c22ed279889c904bcc19883ff8b9
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 22:13:41 2014 +0200

    Fix Jenkins translation jobs

    The job glance-propose-translation-update
    does not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the job.

    Add test to pep8 that checks that no new breakages get
    introduced.

    Change-Id: I1bae1c1bf27bf664b24cf75fc96efb0fcfddc392
    Closes-Bug: #1299349

Changed in glance:
status: New → Fix Committed
Changed in openstack-manuals:
status: New → Fix Committed
Thierry Carrez (ttx) on 2014-04-07
Changed in ceilometer:
milestone: none → icehouse-rc2

Reviewed: https://review.openstack.org/83954
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=faada69444988c66a09ade812e10fa2ab5a0adae
Submitter: Jenkins
Branch: master

commit faada69444988c66a09ade812e10fa2ab5a0adae
Author: Andreas Jaeger <email address hidden>
Date: Sat Mar 29 06:52:48 2014 +0100

    Fix Jenkins translation jobs

    The jobs ceilometer-propose-translation-update and
    ceilometer-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Add test to pep8 that checks that no new breakages get
    introduced.

    Change-Id: I3d7d04a7d5d073d5123f1f7b1e16fd3ef167acfb
    Closes-Bug: #1299349

Changed in ceilometer:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/84207
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=8f5018eec7726112e378f090e119d9d3cbfb598e
Submitter: Jenkins
Branch: master

commit 8f5018eec7726112e378f090e119d9d3cbfb598e
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 20:37:27 2014 +0200

    Check that all po/pot files are valid

    Perform minimal checks on po and pot files so that no broken files can
    get imported.

    Change-Id: I5dd0ca872fa25f8ef9557f039904090c4a672ff3
    Co-Authored-By: Clark Boylan <email address hidden>
    Related-Bug: #1299349

Thierry Carrez (ttx) on 2014-04-07
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-07
Changed in glance:
milestone: none → icehouse-rc2

Reviewed: https://review.openstack.org/83961
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=79fe7a509cd3bebf513fd24939954df53e7d8a52
Submitter: Jenkins
Branch: master

commit 79fe7a509cd3bebf513fd24939954df53e7d8a52
Author: Andreas Jaeger <email address hidden>
Date: Sat Mar 29 09:42:52 2014 +0100

    Fix Jenkins translation jobs

    The jobs nova-propose-translation-update and
    nova-upstream-translation-update do not update from
    transifex since our po files contain duplicate entries where
    obsolete entries duplicate normal entries.

    Remove all obsolete entries to fix the jobs.

    Change-Id: I7f51922472f12f3a13feb71a6fb344076781d7e9
    Closes-Bug: #1299349

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2014-04-08
Changed in ceilometer:
importance: Undecided → Medium
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-08
Changed in glance:
assignee: nobody → Thierry Carrez (ttx)
importance: Undecided → Medium
Thierry Carrez (ttx) on 2014-04-08
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-08
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-09
Changed in heat:
status: Fix Committed → Fix Released
Changed in openstack-manuals:
importance: Undecided → Medium
assignee: nobody → Andreas Jaeger (jaegerandi)
milestone: none → icehouse
Changed in openstack-manuals:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in keystone:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in ceilometer:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in nova:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in cinder:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in heat:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in glance:
milestone: icehouse-rc2 → 2014.1
Thierry Carrez (ttx) on 2014-04-17
Changed in neutron:
milestone: icehouse-rc2 → 2014.1

Reviewed: https://review.openstack.org/84226
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=41870b6e553572ff29965efc417509468995090d
Submitter: Jenkins
Branch: master

commit 41870b6e553572ff29965efc417509468995090d
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 21:39:28 2014 +0200

    Check that all po/pot files are valid

    Perform minimal checks on po and pot files so that no broken files can
    get imported.

    Change-Id: Ib438faddb362eb8e61eaa64a870160196f16c594
    Related-Bug: #1299349
    Co-Authored-By: Clark Boylan <email address hidden>

Reviewed: https://review.openstack.org/84211
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=48394142371702715e7204f4ffb7fd32a2b50a6b
Submitter: Jenkins
Branch: master

commit 48394142371702715e7204f4ffb7fd32a2b50a6b
Author: Andreas Jaeger <email address hidden>
Date: Mon Mar 31 20:54:06 2014 +0200

    Check that all po/pot files are valid

    Perform minimal checks on po and pot files so that no broken files can
    get imported.

    Add msgfmt to list of requirements in doc/source/setup.rst.

    Change-Id: Ifa9fe2fc1ca9a41621ff3039c249700cdb1cd0ac
    Related-Bug: #1299349
    Co-Authored-By: Clark Boylan <email address hidden>

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers