Publisher should set modification times on Releases et al

[LaMont Jones]

Due to caching and such, the slew between Packages/Sources/Releases/Releases.gpg for any given pocket should be the same, so that we can put an "Expires: 30min from timestamp" header into apache, and thereby have a hope that the files all expire at the same time. See RT#52121.

Below is the hack that syncproxy does to implement the fix, with the side effect that all of those files were never current. The diff is the current fix to make it stop hacking files a day after the pocket is published the final time (release). It continues to hack on the files for 1 day after their final publishing, then they magically snap back to the as-published times.

The correct fix is to have the publisher emit the files with the same modification timestamp.

lamont

=== modified file 'mirrorsupported.sh'
--- mirrorsupported.sh 2012-05-29 22:09:58 +0000
+++ mirrorsupported.sh 2012-08-06 16:35:55 +0000
@@ -123,9 +123,10 @@
 echo "$(date -R): Cleaning up day-old filelists to expire stay-of-execution."
 find filelists/ -mtime +1 | xargs rm -fv

+# See RT52121: only touch things that are less than a day old
 CURRENTTIME=$(date +%Y%m%d%H%M)
 find ${DISTDIR} \( -name 'Packages*' -o -name 'Sources*' -o -name 'Release*' \) -print0 | xargs -0 ls -ld > ${ROOTDIR}/project/trace/${HOSTNAME}.dists-timestamps
-find ${DISTDIR} \( -name 'Packages*' -o -name 'Sources*' -o -name 'Release*' \) -print0 | xargs -0 touch -t ${CURRENTTIME}
+find ${DISTDIR} ! -mtime +1 \( -name 'Packages*' -o -name 'Sources*' -o -name 'Release*' \) -print0 | xargs -0 touch -t ${CURRENTTIME}

 echo "$(date -R): Critical mirror operations completed."
 savelog -c 750 $LOGFILE

[Adam Conrad (adconrad)]

And by "emit the files with the same modification timestamp" (which is impossible to do), what lamont means is that we need to post-process in finalize.d to touch Release*/Packages*/Sources* for all the dirty pockets we just published. Should be a pretty trivial patch.

[Colin Watson (cjwatson)]

I think it'd be better to do that post-processing directly in the publisher. I have a patch in the works.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r15836 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15836>.

[Colin Watson (cjwatson)]

While this doesn't break (and thus is deployable), it also doesn't appear to actually fix the bug.

[Colin Watson (cjwatson)]

Actually, the only reason this didn't appear to fix the bug on dogfood was that the call to _syncTimestamps is incorrectly behind a check for whether there's a signing key, so this will work on production. I'll land a follow-up branch to make the fix visible on dogfood too.

[Colin Watson (cjwatson)]

revno: 71
fixes bug: https://launchpad.net/bugs/1033581
committer: Colin Watson <email address hidden>
branch nick: ubuntu-archive-publishing
timestamp: Sat 2015-09-26 02:01:27 +0100
message:
  Make signatures have the same timestamp as the unsigned files, to aid cache configuration.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r17766 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/17766>.