Define a better policy for disabling archive mirrors

Bug #80990 reported by Guilherme Salgado
14
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned

Bug Description

Archive mirrors are currently disabled by the mirror prober only if no content at all is found on them. This is not optimal because we end up including outdated mirrors in the public listings.

After some discussion we agreed to do the following. For now, at least.

1. If there's anything 4 days behind on an archive mirror, that mirror should be disabled.
2. If any files are missing from any released (current and supported) releases on the RELEASE pocket, the mirror should be disabled.

Revision history for this message
Guilherme Salgado (salgado) wrote :

One thing we could easily do is disable all mirrors with an overall freshness "worse" than N days. That'd be trivial and could be a starting point, at least.

Changed in launchpad:
assignee: nobody → salgado
importance: Undecided → Medium
status: Unconfirmed → Confirmed
Revision history for this message
Karl Tilbury (karl-tilbury) wrote :

I think at this point archive mirrors which are over 7 days old should be failed as well as mirrors for any architecture listed as "Unknown freshness".

I assume that "Unknown freshness" means it can not be determined how fresh a mirror is and that it could be older than 7 days anyway? So it seems logical to fail these mirrors too.

description: updated
Revision history for this message
Guilherme Salgado (salgado) wrote :

defered as we'll discuss it during all hands

Changed in launchpad:
milestone: 1.1.11 → 1.2.1
milestone: 1.2.1 → 1.1.12
Changed in launchpad:
milestone: 1.1.12 → none
Changed in launchpad:
milestone: none → 1.1.12
Revision history for this message
Guilherme Salgado (salgado) wrote :

The current heuristic used to guesstimate a mirror's freshness is not good for pockets which are not updated often (e.g. Backports, Proposed), so it often leaves these pockets on a days/weeks behind state. This means lots of mirrors will be disabled if we implement what's discussed on this bug, so we're not doing it until we sort out the other issue.

In order to develop a better heuristic we'd have to perform lots of tests and try several different approaches (which is going to take quite some time), so another option would be to skip all non-essential pockets (release, updates, security, ...) when probing. We may also want to skip the non-essential components (multiverse?).

Changed in launchpad:
milestone: 1.1.12 → 1.2.1
Changed in launchpad:
assignee: salgado → carlos
Changed in launchpad:
milestone: 1.2.1 → 1.2.2
Revision history for this message
Carlos Perelló Marín (carlos) wrote :

I will not have time in this cycle

Changed in launchpad:
assignee: carlos → nobody
Changed in launchpad:
milestone: 1.2.2 → none
Revision history for this message
Gökdeniz Karadağ (gokdeniz) wrote :

A suggestion,
A timestamp file or a dummy timestamp package could be included in all components, the prober would check for the timestamp of that file, which is updated frequently( every six hours ? every hour ?) Gentoo does something like this and it works good. See http://mirrorstats.gentoo.org/

Revision history for this message
Gökdeniz Karadağ (gokdeniz) wrote :

Hi,

Can this issue be handled ?

Here is a sample case; presence of cl-sql-oracle_4.0.4-1_all.deb in karmic koala multiverse sets the whole mirror status in mirror listings to "one week behind"
http://launchpadlibrarian.net/26337480/ftp.linux.org.tr-archive-probe-logfile.txt

affects: launchpad-foundations → launchpad-registry
Curtis Hovey (sinzui)
Changed in launchpad-registry:
importance: Medium → Low
Jonathan Davies (jpds)
tags: added: mirror
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.