binary package filelists have issues on updates and non-x86 arches

[Steve Beattie]

If a binary package did not exist when a release was released, but is later added as part of a security update, the /$RELEASE-updates/ "Download package" section is generated correctly, but the /$RELEASE/ version of the page is not. For example, the linux-hwe source package version 4.13.0-37.42~16.04.1 adds the linux-image-4.13.0-37-generic to the xenial-security and xenial-updates pockets, but it does not exist in the xenial pocket (frozen in time). This results in the following pages:

  https://packages.ubuntu.com/xenial-updates/linux-image-4.13.0-37-generic

and

  https://packages.ubuntu.com/xenial/linux-image-4.13.0-37-generic

The former gets a correctly generated "Download linux-image-4.13.0-37-generic" section; all the arches have entries, and the filelist links all exist. However, for the latter, the "Download linux-image-4.13.0-37-generic" is missing the non-x86 arches and the x86 filelist links lead to 404s:

  https://packages.ubuntu.com/xenial/amd64/linux-image-4.13.0-37-generic/filelist

Additionally, even for packages that existed when the release pocket was frozen, on security updates, the Download section is not reporting the correct versions for non-x86 arches. For example, zsh in xenial-updates looks like:

  https://packages.ubuntu.com/xenial-updates/zsh

but in /xenial/, it looks like:

  https://packages.ubuntu.com/xenial/zsh

where the x86 versions are correctly listed as 5.1.1-1ubuntu2.2, but the non-x86 versions are listing the original xenial release version, 5.1.1-1ubuntu2. And the non-x86 download pages are incorrectly pointing to the 5.1.1-1ubuntu2 versions as well, e.g.:

  https://packages.ubuntu.com/xenial/arm64/zsh/download

[Rhonda D'Vine (rhonda)]

Thanks for the great explanation. That seems to indeed being an issue with "overlay" releases. I'm currently uncertain how to approach this and fix it, but your analysis is correct and helpful.