apt-mirror not getting by-hash directories

[LoOoD]

I see that apt-mirror has been patch to get the files in dep-11 directory but the by-hash subdirectory isn't being mirrored or generated.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[David Venz (david-venz)]

I've been hacking at a copy of apt-mirror to make it more InRelease/by-hash aware. The attached script is *not* fit for general use, but I was able to create a local xenial mirror with it (N.B. it will download ALL sources and ALL binary arches that can be found on the mirror it's pointed at, I have to go back and restore the more selective behaviour the script used to have). Please treat it as 'for study only'!

Script behaviour:
Following the algorithm near the bottom of http://www.chiark.greenend.org.uk/~cjwatson/blog/no-more-hash-sum-mismatch-errors.html :
- The script downloads the InRelease files first, and then attempts to fetch each of the index files listed inside each InRelease. Importantly, it attempts to download their by-hash equivalent file at the same time.
-- Note that this covers dep-11 and translations because they are listed in the InRelease file, so special handling for those files in apt-mirror is removed.
-- Not all mirrors will carry all index files or architectures. The script used to handle this, and continues to do so.
-- The list of index files to fetch from InRelease is probably where the 'better filtering' should be restored in a proper version of this script.
- Then for each successfully fetched index file, it looks to see if its by-hash equivalent is present, and overwrites any normal copy of the file with its by-hash equivalent, unzipping in the process.
- Index files are then processed as normal, collecting further files (debs etc) to download.

Also note that when serving up the local repository, I've found it necessary to configure apache with mod_rewrite rules to serve up e.g. Components-amd64.yml if only Components-amd64.yml.gz exists in the mirror (apt doesn't seem to look for the .gz version first, but I could be mistaken).