Product release finder should import all files

[Michael Kuhn]

The product release finder should import all files for a particular release.
For example, I have sushi-1.0.1.tar.bz2 and sushi-1.0.1.tar.gz, but the release finder only imports the .tar.bz2.

[Curtis Hovey (sinzui)]

Both file types are supported and tests show that multiples will match at the same time. How long has the tar.gz not been found?

The file will not be downloaded if it was is in the Launchpad librarian. Deletion from the librarian is a rare event, so if it was added, then marked for deletion, it will not be downloaded even though you cannot see it.

[Michael Kuhn (suraia)]

This does not seem to be a problem of the finder not finding the .tar.gz:
Looking at the code (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/annotate/head:/lib/lp/registry/scripts/productreleasefinder/finder.py) it seems that only the first matching file is added. All other release tarballs are ignored, see hasReleaseTarball(), addReleaseTarball() and handleRelease().

[Curtis Hovey (sinzui)]

hasReleaseTarball() wrongly assumes that if the release has anything like a code tarball, that there all the downloading is complete. So downloading for a release stops once the first package is downloaded which means release notes and installers are also being ignored.

handleRelease() knows the filename. It should pass it to hasReleaseTarball() to verify if the release already has the file. Rename hasReleaseTarball => hasReleaseFile(self, product_name, series_name, release_name, file_name)

[Curtis Hovey (sinzui)]

Fixed in launchpad devel r9777.

[Curtis Hovey (sinzui)]

Testing reveals that this is not fixed yet. Running the PRF in debug mode I can see an abort is called during addReleaseTarball(). The code has a loop that duplicates the old hasReleaseFile() method to end early if there is a binary associated with the release.

We want to remove the whole loop because hasReleaseFile() makes the decision; addReleaseTarball() should do as it is told. This will also make adding files faster since the the loops not needed.

[Curtis Hovey (sinzui)]

Really fixed in launchpad devel r9827.

[Curtis Hovey (sinzui)]

Fixed released in launchpad-project 3.1.10.