on-cd repo have Release files referencing uncompressed indices, which are not on the cd

[Michael Hudson-Doyle]

For example:

mwhudson@anduril:~/isos$ sudo mount -r ubuntu-20.04.1-live-server-amd64.iso /mnt
mwhudson@anduril:~/isos$ cat /mnt/dists/focal/Release
Origin: Ubuntu
Label: Ubuntu
Suite: focal
Version: 20.04
Codename: focal
Date: Thu, 23 Apr 2020 17:33:17 UTC
Architectures: amd64 i386
Components: main restricted
Description: Ubuntu Focal 20.04
Acquire-By-Hash: yes
SHA256:
 44fa689d816504f1c9ba27b08d46f25a4897a4f21ced7921f5a8530e32ce0b60 95 main/binary-amd64/Release
 16845fa97086e82e6a77c54c0cc5cbfbcc0dd6498639c53e3fda35238bb6e5ad 18748 main/binary-amd64/Packages.gz
 b933d017be7d0dd1866f1c5fd50eb4f8bc530d5ffc3be29ba1bdb39448d8b32f 65067 main/binary-amd64/Packages
 136e603984989c1db83d4cef46881f5b197d65120c49fc718e0a81bb21a166bf 94 main/binary-i386/Release
 a04eed84506e2483b11edfa57d187b2f3558b4e68c08937e3181b6c5f5a823da 3067 main/binary-i386/Packages.gz
 7ae65db95ed836c36c05fc5214326dc7f2764add0500bce68c06e1c2f3462b7e 7683 main/binary-i386/Packages
 d8f70ca1fe1aaab95538a2c4d45118be407c0c479a247a41b2de094041cb940a 101 restricted/binary-amd64/Release
 e7ab72b8f37c7c9c9f6386fb8e3dfa40bf6fe4b67876703c5927e47cb8664ce4 40 restricted/binary-amd64/Packages.gz
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 restricted/binary-amd64/Packages
 f51659fc018cde566f5297af01955e5749d53de3bce71c14e132e02ae6ca4363 100 restricted/binary-i386/Release
 e7ab72b8f37c7c9c9f6386fb8e3dfa40bf6fe4b67876703c5927e47cb8664ce4 40 restricted/binary-i386/Packages.gz
 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 0 restricted/binary-i386/Packages
mwhudson@anduril:~/isos$ ls /mnt/dists/focal/main/binary-amd64/
Packages.gz Release

This seems to have been the case forever and affect all flavours (OK, I only checked trusty GA and live server, legacy server and desktop but that seems close enough).

[Dimitri John Ledkov (xnox)]

oh oh oh I know!

We have very dodgy apt-ftparchive config, cause we used apt-ftparchive which may or may not support all the compressions, and then we patch in and generate Release file by hand.

There is some real dodge code in debian-cd, which has been reworked a long time ago.

I think now that cdimage is on a more modern system, we can call it directly to generate actually valid Release & InRelease files.

[Dimitri John Ledkov (xnox)]

reworked -> as in upstream in debian; rather than in the ubuntu-cdimage's debian-cd.

[Michael Hudson-Doyle (mwhudson)]

Seems the culprit code is in tools/add_secured.

I don't have more than the faintest idea of how the flow of creating the pool goes.

I know there is code in ubuntu-cdimage for choosing which seeds to include and germinating them into a list of packages. Presumably something then downloads these packages, arranges them in to a pool like directory tree and runs apt-ftparchive? Or does apt-ftparchive's job by hand, it seems like?

tools/add_secured has this comment

# Only keep Packages.gz; it's a waste of space to keep Packages too. We
# delete these here rather than earlier because (for now) debootstrap
# and apt *do* need Packages to be mentioned in the Release file.

which sounds relevant to the problem at hand, but the "rather than earlier" bit makes no sense at all in the context of the /cdrom/pool.

[Colin Watson (cjwatson)]

I'm fairly sure that apt interprets the checksums for the uncompressed Packages files as "this is what we expect you to see if you uncompress the file" rather than "this is expected to actually exist on disk"; at least I remember it used to be that way.