Feature request: report missing packages on check-packages action

[Paul Goins]

One bit of information which could add extra confidence re: cleaning packages in the mirror, and fairly easy to add based upon the current code, is information on packages which are listed in the Packages files but not present in the mirrored repository, i.e. missing packages.

In charm.py, AptMirrorCharm._check_packages(), we have this section of code here:

        packages_to_be_removed = existing_packages - indexed_packages
        freed_up_space = 0
        for p in packages_to_be_removed:
            freed_up_space += p.stat().st_size
        return packages_to_be_removed, convert_bytes(freed_up_space)

It would be trivial to add a calculation of missing packages, e.g.:

        packages_to_be_removed = existing_packages - indexed_packages
        missing_packages = indexed_packages - existing_packages
        freed_up_space = 0
        for p in packages_to_be_removed:
            freed_up_space += p.stat().st_size
        return packages_to_be_removed, missing_packages, convert_bytes(freed_up_space)

This seems like information that could be exposed to the user via the check-packages action, or perhaps a separate action if preferred.

Why do I think this is needed? The idea is that this gives extra confidence that all the packages in the repository's Packages* files are indeed actually present and accounted for.

During personal testing with code derived from this charm (with the above changes as well), I've seen one mirror where missing packages were reported - this turned out to be a broken pool symlink in one of the snapshots, which I was able to identify and address, and then afterwards I was able to confirm that the repository data was healthy - that we indeed could find all the packages that were supposedly in each of the Packages files.

[Eric Chen (eric-chen)]

Migrate to https://github.com/canonical/charm-apt-mirror/issues/33