checksum-directory does not re-use checksums from olddirs

Bug #1815592 reported by Łukasz Zemczak
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu CD Images
Triaged
High
Łukasz Zemczak

Bug Description

The bin/checksum-directory script is intended to create checksums of all images in a given directory. The script itself offers a functionality of adding additional 'old directories' where it should look for existing checksums for images in the new directory and try re-using those - instead of re-checksumming the files all over again (to save time). This doesn't seem to work.

The checksum-directory merging of checksums works in such a way that each entry in the current directory is checked to see if it can be re-used from one of the olddirs. First there's a check for symlinks. If the entry is not a symlink, then the next method of 'checking' is checking the timestamps of the files against the checksum-storage files. The current design of the timestamp checking makes it basically impossible to re-use checksums from other directories though.

Each image file is checked for max(st.st_mtime, st.st_ctime) and then compared with the old directory's checksums file (for instance, MD5SUMS) mtime (entry_time > dir_time). Every move, hard-link or copy operation of an image file modifies at least one of the timestamps (usually st_ctime), resulting in checksum-directory not considering re-using the checksum. I think only mtime should be checked?

Related branches

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

I'd like to know if we're actually using the olddirs functionality right now, other than for removing old releases. Once we know that, I would propose using only st.st_mtime for the timestamp check. Checksums only care about the *contents* of the image, not the metadata. Move and link operations will only touch the st_ctime timestamp, so it would allow re-using the checksums in cases of image moves. Also, image moves and hard-links should be fairly safe.

I'll prepare a PR but leave it open for discussion.

Changed in ubuntu-cdimage:
assignee: nobody → Łukasz Zemczak (sil2100)
status: New → In Progress
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

For the archival script I'm currently working around it by touching the *SUMS files after performing the move operation. But I think the implementation should really just look at mtime. Any reason for it to have to consider ctime?

description: updated
Changed in ubuntu-cdimage:
status: In Progress → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.