mirror-probe: "up to date check" is too fast

[ftp.halifax.rwth-aachen.de]

(I did not find a way to file a bug against the mirror-probe system, please move accordingly)

Dear all,

on the mirror status page (e.g. https://launchpad.net/ubuntu/+mirror/ftp.halifax.rwth-aachen.de-archive) I see information like "six hours behind" and "up to date".

Based on my observerations in the last few months, I sometimes noticed my mirror is regarded as not-up-to-date, although we synchronize a) with the official ubuntu rsync master mirror b) as often as it is stated in the wiki page on how to set up mirrors ("Keep up to date. Please try to mirror about four times a day (so every six hours) for archive mirrors. ").

You seem to check mirrors for files that were included on your master mirror only minutes or even seconds ago. In this case it should be fairly obvious that the file is not (yet) available on (non-pushed) mirrors.

I suggest to not check for files newer than six hours or let the mirrors synchronize more often than every six hours.

Thanks,
Carsten

[ftp.halifax.rwth-aachen.de (ftp-halifax)]

Any news on this?

[Richard Marshall (ademccade)]

Dußeldorfkeit mann platzerien könnt haben, und zweitag für deinem gerhabtschall scheiften mir. Darfunhötte die penzilinkäbel bitte.

[ftp.halifax.rwth-aachen.de (ftp-halifax)]

Dear Ubuntu folks, I raised this issue 12 years ago. Do you care about mirrors?

[Haw Loeung (hloeung)]

Unlikely it's checking for files only published minutes or seconds ago but someone from the Launchpad team might want to check and confirm this.

Also, if it's missing files from an hour or two ago, then it's like as it says "not-up-to-date".

You can also apply to have your mirror set up as a "push mirror" where it's part of the update process and is triggered to be updated when there are changes. See:

| https://wiki.ubuntu.com/Mirrors/PushMirroring

Finally, if I understand correctly, the "up to date" or not is just a status message?

Looking at your mirrors' page, I see this:

"""
Last probe
This mirror was last verified 15 hours ago.
"""

I would say not "too fast" if you ask me. In any case, I'll leave it for the Launchpad team to investigate and make improvements as they see fit on this front.

[ftp.halifax.rwth-aachen.de (ftp-halifax)]

How can a mirror that complies with the mirror setup rules be out of date? You missed to comment on this crucial fact. Push mirroring is not mandatory, as far as I'm aware. I don't know if the "up to date" is a string that is ignored or if it has some Ubuntu-internal consequences - you might want to ask Ubuntu folks about this. I do know, however, that it causes confusion and unnecessary work for me (and other mirror admins) who try to figure out what's wrong when, in fact, nothing is wrong.

The issue is not how often the mirror is verified, but how long it takes between having a new file in the checker's requirements and checking for said file. This could have happened 15 hours ago, for all we know.

Again, which is why I created this bug, I would say that it is "too fast" if it happens repeatedly for correctly functioning mirrors.

[Haw Loeung (hloeung)]

I wouldn't call it rules, but guidelines. The way I read it is, at least a minimum of 4 times a day, or recommended 4 times a day, but sure, I'll take your word on it. I've brought up an internal discussion about it.

IIUC, it's a status message for people in the community to decide on which is closest and best when selecting an APT source to use overriding the default that is set (archive.ubuntu.com or ccTLD.archive.ubuntu.com). It may be used for source selection during the installation process. But security.ubuntu.com is also included as a source so servers/VMs/machines would still get the latest packages.

This is for the main Ubuntu Archive. For releases or cdimages, it may be used for ISO downloads from the main Ubuntu website's download section. It used to in the past but that may have changed.

Again, I've brought it to the attention of the teams involved internally to hopefully get some traction on this front.

[Clinton Fung (clinton-fung)]

The purpose of this status is to provide users with an indication of the freshness of the data in the mirror. This indication of freshness is not related to compliance with rules, nor is it intended to call into question a mirror's ability to remain up to date.

I take these reports seriously, and my current line of thinking is that we should more transparently surface what we know about when the last synchronisation occurred, rather than interpret that information as "Up to date" or not.

[ftp.halifax.rwth-aachen.de (ftp-halifax)]

Thank you. Please also take into account that there might not be a recent push. In the case of ftp.halifax.rwth-aachen.de, updates have been disabled for a month or so, but several old Ubuntu versions show up as "up to date" (which is correct, in some sense).