wanted: get rid of SBCL_HOME and run-sbcl.sh -- too much hassle

[Attila Lendvai]

issue:
-------
 - SBCL_HOME is cumbersome, misconfiguration results in seemingly unrelated errors leading to long headaches

 - run-sbcl.sh interferes with argv parsing

notation:
-----------
 - in this document 'ROOT' means the checked out directory of the SBCL git repository

suggested solution:
------------------------
 - make.sh should put all the install-ready files into ROOT/build (or somesuch, but don't conflate it with the current temporary directory called 'output').

the files needed for running sbcl are:
   - all the built contribs (currently ROOT/obj/sbcl-home/contrib/)
   - sbcl.core (currently ROOT/output/sbcl.core)
   - sbcl executable (currently ROOT/src/runtime/sbcl)

- get rid of run-sbcl.sh, people should run ROOT/build/sbcl directly

 - if SBCL_HOME is not set, then initialize the internal variables to the pwd of the sbcl executable (resolve symlinks or not?)

 - never set SBCL_HOME (maybe even get rid of it?)

consequences:
-------------
 - with this setup the traditional unix way of installing (the binary and the core in a separate directory) will not work without either setting SBCL_HOME, or trying the standard unix path's.

 - the windows installer could install sbcl into any directory and it would just work.

 - checked out and built repos would just work, making it easy to keep a few different sbcl's around

[Nikodemus Siivola (nikodemus)]

I basically agree.

This couples nicely with my plan for new installation where /usr/local/bin/sbcl -> /usr/local/lib/sbcl/<version>/sbcl-runtime or equivalent.

[Attila Lendvai (attila-lendvai)]

FYI, the sad reality out in the wild, people struggling due to SBCL_HOME:

http://permalink.gmane.org/gmane.lisp.slime.devel/10464

[Christophe Rhodes (csr21-cantab)]

Attila Lendvai <email address hidden> writes:

> FYI, the sad reality out in the wild, people struggling due to
> SBCL_HOME:
>
> http://permalink.gmane.org/gmane.lisp.slime.devel/10464

For the record, the issue described in this thread was completely
unrelated to SBCL_HOME.

Christophe

[Attila Lendvai (attila-lendvai)]

sorry for just wishes and no patches, but it would be nice if the build was also restructured. run-sbcl.sh is still present and is necessary when sbcl is built from source and is not installed:

$ ./src/runtime/sbcl
fatal error encountered in SBCL pid 5901(tid 0x7fb772baa280):
Can't find core file relative to ./src/runtime/

ideally, it should just run as is when started from a build/ directory.

[Pouar (pouar)]

This fix seems to be causing ASDF to output fasl files in the current directory when the image is dumped as an executable
https://gitlab.common-lisp.net/asdf/asdf/issues/28

[Pouar (pouar)]

Is this intended behavior?

[Stas Boukarev (stassats)]

No idea how ASDF misuses SBCL internals for finding output locations.

[Robert P. Goldman (rpgoldman)]

Follow up: Here's what goes wrong with ASDF:

We have to NOT move output fasls from the lisp implementation home directory. This is necessary so that people don't recompile hunks of lisp code distributed with SBCL and end up with the fasls put somewhere inappropriate.

This means that the first rule of any output translations ruleset says (effectively) DON'T redirect fasls from the lisp implementation directory.

Unfortunately, if the lisp implementation directory is the directory that an executable is started in, AND if a developer starts up an executable in the source directory where it was built, placement of fasl files for that source directory will get messed up.

I'm not sure why pretending an executable's directory is the lisp installation directory is helpful. Wouldn't it be less misleading to set the lisp installation directory for dumped executables to NIL, because there is no valid installation directory, in general, for a dumped executable, which won't have the expected suite of installed files in the usual place?

[Stas Boukarev (stassats)]

One of the possible locations for finding contribs/ and sbcl.core is beside the executable.

[Stas Boukarev (stassats)]

I appended contrib/ to sbcl-homedir-pathname. Hope nothing, besides ASDF, uses it...

[Pouar (pouar)]

appending contrib/ to sbcl-homedir-pathname seems to have fixed it

[Pouar (pouar)]

ok, apparently https://github.com/sbcl/sbcl/commit/71f7507fc119b4c88b428d71e7aaf0ded918558e causes it to places fasls in projects that have a contrib directory. I guess putting ./ before contrib/ makes a difference.

[Stas Boukarev (stassats)]

But there is ./ before contrib/. And it even returns NIL if there's no contrib/ directory.

[Stas Boukarev (stassats)]

I guess it gets foiled by *default-pathname-defaults* being changed.

[Pouar (pouar)]

ok, https://github.com/sbcl/sbcl/commit/7464a809f01f fixed it

[Attila Lendvai (attila-lendvai)]

this change has broken our setup.

my ultimate goal with this issue was to be able to run a compiled sbcl straight from its directory, without installing it, and without the use of run-sbcl.sh.

IIUC, now even run-sbcl.sh cannot communicate SBCL_HOME to the executable anymore (the breakage).

what's needed is a reorganization of the build process so that the resulting files are in an appropriate directory structure for the sbcl executable to find what it needs.

ideally, into an encompassing build/ directory.

[Stas Boukarev (stassats)]

run-sbcl.sh is working fine, what do you mean?

[Attila Lendvai (attila-lendvai)]

i got confused. whatever causes the breakage is deeper (it's in the process of capturing sbcl's home directory at build-time into a dumped executable image).

sorry for the noise! i'll get back after more investigation -- if needed.