high load in script server for lp_publish jobs

Bug #954062 reported by JuanJo Ciarlante
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Unassigned

Bug Description

Our internal server that run lp_publish jobs has been experiencing a highload "seasons" usually 1~2 per week, which doesn't directly correlate to specific cron entries.

I'd suggested "shaping" the likelihood of scripts being run from cron based on current loadavg, as e.g.
load==10 -> P=1.0 down to load==21 -> P=0.0, by prefixing them with:
 L=$(awk '{ printf ("%0.0f\n",$2) }' /proc/loadavg);test $((RANDOM % 10 > (20-L))) -eq 0 && ...

, but ~lifeless pointed out that he had observed places for improvement in the scripts being run:
- * * * * * ... launchpad/cronscripts/generate-ppa-htaccess.py
- */5 * * * * ... launchpad/cronscripts/publishing/cron.publish-ppa ...
- */20 * * * * ... launchpad/cronscripts/ppa-generate-keys.py ...
- 3,33 * * * * ... launchpad/scripts/process-death-row.py ...
- 59 05 * * 0 ... launchpad/cronscripts/publishing/cron.daily-ppa

JuanJo Ciarlante (jjo)
tags: added: canonical-losa-lp
Changed in launchpad:
status: New → Triaged
importance: Undecided → Critical
Revision history for this message
Robert Collins (lifeless) wrote :

(Not a regression / escalated at this point)

Changed in launchpad:
importance: Critical → High
William Grant (wgrant)
Changed in launchpad:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.