Feature request: Zstd / lz4 compression

[Pierre Neidhardt]

Currently the deflate (zlib) compression that's used by SBCL has one rather big downside: it slightly increases startup time. This is bound to happen because SBCL executable are always big.

A better option would be to use the lz4 or Zstd algorithms:
https://en.wikipedia.org/wiki/Zstandard

With such an algorithm, the startup time different might not be noticeable.

While on the topic, it could be useful to explain how to enable compression with ASDF in the manual:

#+sb-core-compression
(defmethod asdf:perform ((o asdf:image-op) (c asdf:system))
  (uiop:dump-image (asdf:output-file o c)
                   :executable t
                   :compression t))

[Stas Boukarev (stassats)]

Does it increase start up time because of the compression algorithm or because with compression the core file can't simply be mmaped?

[Pierre Neidhardt (ambrevar)]

I assume the compression algorithm is at play here, but I'm not sure how
to measure this.

[Orivej Desh (orivej)]

zlib indeed made sbcl startup unacceptably slow for my use case, and switching to lz4 made it fast enough. This was discussed in the "Interest in patch incorporating lz4?
" thread: https://sourceforge.net/p/sbcl/mailman/message/32989588/ and the patch is still available (although not updated) in https://github.com/orivej/sbcl/commit/lz4-squashed . I have not tested zstd at the time, but if sbcl was to add one new algorithm now, zstd probably suits it better.

[Pierre Neidhardt (ambrevar)]

Excellent, thanks a lot, this is an amazing new addition to SBCL!