On Wed, Jun 26, 2024 at 04:33:40PM -0000, Simon Déziel wrote:
> @racb, thank you very much for your analysis in
> https://lists.ubuntu.com/archives/ubuntu-devel/2024-May/043006.html. In
> there, you said:
>
> > Stripping older hashes reduces download size by around 35% to 40%.
>
> Which I think meant you dropped the `md5` and `sha1` sums, right? If so,
> the linked branch
> (https://code.launchpad.net/~xnox/launchpad/+git/launchpad/+ref/only-
> sha256) proposed even more savings by only keeping `sha256` sums which
> is what most of the ecosystem seems to depend on.
Right - I kept only SHA512 for the most pessimistic analysis. SHA256
only would be better but I didn't analyse that.
> Also in your ML post you said:
> While 2.4MiB might seem small, this is not cache-able by the client
> (max-age=0), only by a potential intermediate cache (s-maxage):
>
> $ wget -qSO /tmp/f http://security.ubuntu.com/ubuntu/dists/jammy-updates/InRelease 2>&1 | grep Cont
> Content-Length: 128049
> Cache-Control: max-age=1420, s-maxage=3300, proxy-revalidate
>
> Even in the case of an intermediate cache, if a newer file is needed, it
> will be downloaded in full so smaller files are better. In other words,
> there is no way to download deltas.
Isn't there a distinction between revalidation and a full download here?
AIUI, caching the file remains possible as long as the content hasn't
changed; it's just that the client is supposed to check that it hasn't
changed which requires query to the origin server. So from a caching
perspective there would be no difference to cache hits if the file is
bigger, and caching the _content_ is still possible. Have I missed
something?
However the file does change fairly frequently. For jammy-updates I'm
not sure if that's every time the Launchpad publisher runs or every time
something new is published into jammy-updates. In those cases a
re-download of the slightly larger file would be required - yes.
On Wed, Jun 26, 2024 at 04:33:40PM -0000, Simon Déziel wrote: /lists. ubuntu. com/archives/ ubuntu- devel/2024- May/043006. html. In /code.launchpad .net/~xnox/ launchpad/ +git/launchpad/ +ref/only-
> @racb, thank you very much for your analysis in
> https:/
> there, you said:
>
> > Stripping older hashes reduces download size by around 35% to 40%.
>
> Which I think meant you dropped the `md5` and `sha1` sums, right? If so,
> the linked branch
> (https:/
> sha256) proposed even more savings by only keeping `sha256` sums which
> is what most of the ecosystem seems to depend on.
Right - I kept only SHA512 for the most pessimistic analysis. SHA256
only would be better but I didn't analyse that.
> Also in your ML post you said: security. ubuntu. com/ubuntu/ dists/jammy- updates/ InRelease 2>&1 | grep Cont
> While 2.4MiB might seem small, this is not cache-able by the client
> (max-age=0), only by a potential intermediate cache (s-maxage):
>
> $ wget -qSO /tmp/f http://
> Content-Length: 128049
> Cache-Control: max-age=1420, s-maxage=3300, proxy-revalidate
>
> Even in the case of an intermediate cache, if a newer file is needed, it
> will be downloaded in full so smaller files are better. In other words,
> there is no way to download deltas.
Isn't there a distinction between revalidation and a full download here?
AIUI, caching the file remains possible as long as the content hasn't
changed; it's just that the client is supposed to check that it hasn't
changed which requires query to the origin server. So from a caching
perspective there would be no difference to cache hits if the file is
bigger, and caching the _content_ is still possible. Have I missed
something?
However the file does change fairly frequently. For jammy-updates I'm
not sure if that's every time the Launchpad publisher runs or every time
something new is published into jammy-updates. In those cases a
re-download of the slightly larger file would be required - yes.