Handle SOURCE_DATE_EPOCH for SquashFS

Bug #1890046 reported by Anatoli Babenia on 2020-08-02
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Snapcraft
Medium
Unassigned
snapd
Medium
Unassigned

Bug Description

For reproducible snap builds is needed that if SOURCE_DATE_EPOCH is set, the timestamps inside of .snap archive were set from it.

--- v2.snap
+++ yakshaveinc_eternal_amd64.snap
├── unsquashfs -s {}
│ @@ -1,9 +1,9 @@
│ Found a valid SQUASHFS 4:0 superblock
│ -Creation or last append time Sat Aug 1 18:36:27 2020
│ +Creation or last append time Sun Aug 2 15:04:00 2020
│ Filesystem size 576 bytes (0.56 Kbytes / 0.00 Mbytes)
│ Compression xz
│ Block size 131072
│ Filesystem is exportable via NFS
│ Inodes are compressed
│ Data is compressed
│ Uids/Gids (Id table) are compressed
├── unsquashfs -d -lls {}
│ @@ -1,6 +1,6 @@
│ Parallel unsquashfs: Using 4 processors
│ 1 inodes (1 blocks) to write

│ drwxr-xr-x root/root 27 2020-05-26 13:34
│ drwxr-xr-x root/root 32 2020-05-26 13:34 /meta
│ --rw-r--r-- root/root 275 2020-08-01 18:36 /meta/snap.yaml
│ +-rw-r--r-- root/root 275 2020-08-02 15:04 /meta/snap.yaml

Ian Johnson (anonymouse67) wrote :

can you try using `unsquashfs` on the snap you get and then `snap pack`? Does `snap pack` work with the `SOURCE_DATE_EPOCH` env var?

Anatoli Babenia (abitrolly) wrote :

Nope, setting SOURCE_DATE_EPOCH for `snap pack` has no effect.

Changed in snapcraft:
status: New → Confirmed
status: Confirmed → Triaged
importance: Undecided → Medium

I see this happen to. I think the problem is that mksquashfs from the snapd snap (or the core snap is too old).

This is from strace when invoking `snap pack`:

[pid 57453] execve("/var/lib/snapd/snap/snapd/current/lib/x86_64-linux-gnu/ld-2.23.so", ["/var/lib/snapd/snap/snapd/current/lib/x86_64-linux-gnu/ld-2.23.so", "--library-path", "/var/lib/snapd/snap/snapd/current/usr/local/lib:/var/lib/snapd/snap/snapd/current/lib/x86_64-linux-gnu:/var/lib/snapd/snap/snapd/current/usr/lib/x86_64-linux-gnu", "/var/lib/snapd/snap/snapd/current/usr/bin
/mksquashfs", ".", "/home/maciek/work/canonical/image/snapd_2.45.3.1+git2336.g484f47d_amd64.snap", "-noappend", "-comp", "xz", "-no-fragments", "-no-progress", "-wildcards", "-ef", "/tmp/.snap-pack-exclude-981155965", "-all-root", "-no-xattrs"], ["DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus", "DISPLAY=:0", "HOME=/home/maciek", "INVOCATION_ID=6b2d474429d64baebb7b8457e0057
a7e", "JOURNAL_STREAM=8:39754", "LANG=en_US.UTF-8", "LC_TIME=pl_PL.UTF-8", "LOGNAME=maciek", "MAIL=/var/spool/mail/maciek", "MANAGERPID=1483", "PATH=/usr/local/bin:/usr/bin:/var/lib/snapd/snap/bin:/home/maciek/bin:/home/maciek/.local/bin:/usr/local/sbin:/var/lib/flatpak/exports/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl", "PLASMA_USE_QT
_SCALING=1", "PWD=/home/maciek/work/canonical/image", "SHELL=/usr/bin/zsh", "SSH_AUTH_SOCK=/run/user/1000/keyring/ssh", "TERM=screen-256color", "TMUX=/tmp//tmux-1000/default,1557,0", "TMUX_PANE=%8", "USER=maciek", "XAUTHORITY=/run/user/1000/gdm/Xauthority", "XDG_DATA_DIRS=/home/maciek/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share/:/usr/share/:/
var/lib/snapd/desktop", "XDG_RUNTIME_DIR=/run/user/1000", "SHLVL=1", "OLDPWD=/home/maciek", "KWIN_TRIPLE_BUFFER=1", "HG=/usr/bin/hg", "MOZ_PLUGIN_PATH=/usr/lib/mozilla/plugins", "EDITOR=emacsclient -t -a ''", "PAGER=less", "LC_MESSAGES=", "_=/usr/bin/strace", "SOURCE_DATE_EPOCH=1596107149"]) = 0

Clearly the environment variable is passed to mksquashfs. However, the mksquashfs binary is not aware of SOURCE_DATE_EPOCH at all:

$ strings /var/lib/snapd/snap/snapd/current/usr/bin/mksquashfs| grep SOURCE_DATE

Now my local binary is:

$ strings /usr/bin/mksquashfs| grep SOURCE_DATE
SOURCE_DATE_EPOCH
FATAL ERROR:SOURCE_DATE_EPOCH and command line options can't be used at the same time to set timestamp(s)
Env Var SOURCE_DATE_EPOCH has invalid time value

We would need to update the mksquashfs that goes into the snap binary for this to work. The version we currently have comes from 16.04.

Changed in snapd:
status: New → Triaged
importance: Undecided → Medium
Anatoli Babenia (abitrolly) wrote :

Need at least version 4.4

https://lore.kernel<email address hidden>/

Jamie Strandboge (jdstrand) wrote :

The squashfs-tools in xenial are patched to support -fstime which I think could be used to achieve the same result. Eg, snapd, if SOURCE_DATE_EPOCH is set and verified, add '-fstime $SOURCE_DATE_EPOCH' to the mksquashfs arguments.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers