Comment 5 for bug 2034656

Revision history for this message
Junien F (axino) wrote : Re: Update Ubuntu ESM cache only once daily?

Grant, thanks for your reply. I'll add one datapoint here, which I would say is our biggest concern right now : a lot of clients still run their daily APT activities (I presume through unattended-upgrades) at cron.daily time, ie 06:25.

This causes a massive storm of traffic on the ESM servers every day at that time, and depending on whether there are actual package upgrades available or not, can cause the servers to be overloaded.

This has been the case for the last 3 days, and looking at the logs on one server during the cron.daily spike we can see that the vast majority of the traffic is client downloading linux-azure-5.4 version 5.4.0-1116.123~18.04.1, from bionic-infra-security.

Now my understanding was that we had worked around this problem in general for APT (i.e. for archive.ubuntu.com and security.ubuntu.com) in bionic by having two different timers for APT activities : one that essentially does "apt update + apt upgrade --download-only" (the tasks that interact with the archive servers), spread over 12h - that's apt-daily.timer. And another timer that does "apt upgrade" at around 06:25 - that's apt-daily-upgrade.timer

Given the bionic-infra-security spike mentioned above, it's clear that something is not respecting these timers and doing "apt update + apt upgrade" at cron.daily time. Would you have any idea what it could be ?

Thanks