Investigate parallelising cron.germinate by arch

Bug #1248867 reported by Adam Conrad
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-archive-publishing
Fix Released
Low
Colin Watson

Bug Description

It might be nice, assuming germinate isn't entirely I/O bound, to parallelise it a bit in the publisher. A quick discussion on IRC came to the conclusion that parallelising by flavour/arch combination wouldn't work so well, but parallelising purely per arch would probably work fine, and could cut the run time down a bit.

Related branches

Adam Conrad (adconrad)
Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
William Grant (wgrant)
Changed in launchpad:
importance: Undecided → Low
status: New → Triaged
tags: added: soyuz-publish
Colin Watson (cjwatson)
Changed in launchpad:
status: Triaged → In Progress
Revision history for this message
Colin Watson (cjwatson) wrote :

We can indeed safely parallelise by architecture. Experimentation with a cloned-and-hacked version of generate-extra-overrides on snakefruit, which has very similar specs to pepo, suggests that this takes us from 162 seconds to 55 seconds (median of five runs in both cases). That's about a 3x improvement rather than the nearly 6x (for six architectures) that we'd hoped for, but still not to be sneezed at. I'll see about making the corresponding changes to generate-extra-overrides.

Colin Watson (cjwatson)
affects: launchpad → ubuntu-archive-publishing
Colin Watson (cjwatson)
Changed in ubuntu-archive-publishing:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.