checksum differences for source package recipe build tarballs

[Jelmer Vernooij]

source package recipes currently produce slightly different tarballs, even for builds with the same manifest. This is caused by the fact that file timestamps in the checkout are different between different runs.

It'd be great if the same timestamp could be used so the tarball checksums match and the build becomes reproducible given a particular manifest. Perhaps the timestamp of the revision that last changed a file could be used for that files' mtime rather than the current time?

'bzr export' already can do similar things like this using --per-file-timestamps.

[Paul "LeoNerd" Evans (leonerd)]

+1 to this.

I see it's not been touched in nearly 6 years. Is there any way I can help with this?

I suspect an easy fix is to just pack files using the timestamp of the most recent commit (or at least, the commit that's being packed into a tarball) instead of "now". Is there code somewhere I could send a patch against?

[Colin Watson (cjwatson)]

https://code.launchpad.net/bzr-builder would be the place to try to start with this, and also https://code.launchpad.net/git-build-recipe which exists now and should be kept up to date with any changes to bzr-builder. Thanks for offering to help, as we're working mostly to capacity at the moment and it's unlikely that this sort of bug will be addressed otherwise.

Most-recent-commit of any of the various branches that form part of the recipe is probably an OK metric to use, although it would be nice if we could also take into account the last modification time of the recipe itself. That latter is more work. I think the most elegant way would be (1) have bzr-builder/git-build-recipe get that modification time from the mtime of the recipe file passed to them as input; (2) have lp:launchpad-buildd lpbuildd/sourcepackagerecipe.py set that mtime from an option passed to it in its build arguments; (3) have lp:launchpad lib/lp/code/model/recipebuilder.py set that argument from self.build.recipe.date_last_modified.

In the first pass, I'd recommend tackling only bzr-builder/git-build-recipe, and proving that the change works by way of tests. If you take my suggested approach of taking the recipe's mtime into consideration, then it won't do anything useful initially in a Launchpad context because the recipe file will always be current, but the other bits can be stitched in afterwards.

Feel free to discuss this with us as noted in https://dev.launchpad.net/Help.