rosetta-pofile-stats script too slow

Bug #361844 reported by Данило Шеган
4
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Данило Шеган

Bug Description

cronscripts/rosetta-pofile-stats.py script is simply too slow, especially with message sharing switchover code.

We've got around 685k POFiles, and in 26h we managed to process only around 182759 files. Optimizations will have to happen in POFile.updateStatistics() method, and might need to be a bit more creative. It's likely slower due to the fix of bug 172780 at the same time, but it's also migrated to use existing code as provided by POFile filtering methods (so we have less code to maintain).

This means that queries can do with more optimizations, but also that we can be a bit more creative in optimizing this: in general, there should not be any need to run full pofile update statistics across entire DB: we can run the full version once a month or so, and only run each day update over files which have been touched in the last 2 or 5 or 7 days.

Changed in rosetta:
assignee: nobody → Данило Шеган (danilo)
importance: Undecided → High
milestone: none → 2.2.4
status: New → Triaged
Revision history for this message
Данило Шеган (danilo) wrote :

Not yet fixed, we should check up on how's it doing.

Changed in rosetta:
status: Triaged → Fix Released
status: Fix Released → Triaged
milestone: 2.2.4 → 2.2.5
Revision history for this message
Данило Шеган (danilo) wrote :

This seems to not be so bad anymore with a workaround for bug 369748 in.

Changed in rosetta:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.