Improve mksquashfs compression

[Simon Déziel]

When snapcraft calls `mksquashfs`, it doesn't specify any block size so it defaults to 131072 (128K). Tweaking this to 1M leads to better compression:

$ sudo mksquashfs /snap/core22/current/ core22.snap -noappend -no-progress -comp xz -no-fragments

$ sudo mksquashfs /snap/core22/current/ core22-b1M.snap -noappend -no-progress -comp xz -no-fragments -b 1M

$ ll -h *.snap
-rw-r--r-- 1 root root 72M Oct 4 15:23 core22-b1M.snap
-rw-r--r-- 1 root root 74M Oct 4 15:22 core22.snap

We can observe similar savings with lxd snap:

$ sudo mksquashfs /snap/lxd/current/ lxd.snap -noappend -no-progress -comp xz -no-fragments -all-root -no-xattrs
$ sudo mksquashfs /snap/lxd/current/ lxd-b1M.snap -noappend -no-progress -comp xz -no-fragments -all-root -no-xattrs -b 1M

$ ll -h *.snap
-rw-r--r-- 1 root root 170M Oct 4 15:29 lxd-b1M.snap
-rw-r--r-- 1 root root 182M Oct 4 15:31 lxd.snap

Another tweak that might be worth it for snaps is to use the `branch/call/jump (BCJ) filter` feature of XZ:

$ sudo mksquashfs /snap/core22/current/ core22-b1M-Xbcj.snap -noappend -no-progress -comp xz -no-fragments -b 1M -Xbcj x86,arm,powerpc
$ sudo mksquashfs /snap/lxd/current/ lxd-b1M-Xbcj.snap -noappend -no-progress -comp xz -no-fragments -all-root -no-xattrs -b 1M -Xbcj x86,arm,powerpc

$ ll -h *.snap
-rw-r--r-- 1 root root 72M Oct 4 15:23 core22-b1M.snap
-rw-r--r-- 1 root root 70M Oct 4 15:38 core22-b1M-Xbcj.snap
-rw-r--r-- 1 root root 74M Oct 4 15:22 core22.snap
-rw-r--r-- 1 root root 170M Oct 4 15:36 lxd-b1M.snap
-rw-r--r-- 1 root root 163M Oct 4 15:35 lxd-b1M-Xbcj.snap
-rw-r--r-- 1 root root 182M Oct 4 15:31 lxd.snap

Note: changing the block size provides almost no gain when using lzo (which doesn't support BCJ filters):

$ sudo mksquashfs /snap/core22/current/ core22-lzo.snap -noappend -no-progress -comp lzo -no-fragments
$ sudo mksquashfs /snap/core22/current/ core22-lzo-b1M.snap -noappend -no-progress -comp lzo -no-fragments -b 1M

$ ll -h core22-lzo*.snap
-rw-r--r-- 1 root root 98M Oct 4 15:39 core22-lzo-b1M.snap
-rw-r--r-- 1 root root 99M Oct 4 15:39 core22-lzo.snap

[Callahan Kovacs (mr-cal)]

@sergio-j-cazzolato - I think snapd is the right category.

Snapcraft doesn't directly call `mksquashfs`. Instead, it calls `snap pack` and snapd makes the call to `mksquashfs`.

[Simon Déziel (sdeziel)]

I filled it under snapd as the calling code is there https://github.com/snapcore/snapd/blob/master/snap/squashfs/squashfs.go#L527-L556

[Simon Déziel (sdeziel)]

@ridwaninvestmen, I'll assume that was an accident to mark this bug as private as there really isn't anything private nor security related here :)