loading contribs without asdf at runtime

[Faré]

I'd like the ability for contribs to be each distributed as a single .fasl file.

Here is a patch that does it.

The catch is that this version gives lowest priority to the contribs, rather than topmost. This can be fixed in a further iteration, but the hard work is done. I had to update ASDF for that, and SBCL should probably wait for the next ASDF release (as well as SBCL unfreeze) to commit, but for the record, here it is.

[Faré (fahree)]

Much improved patch. Removes the magic that appends code to asdf, so SBCL uses a pristine asdf.lisp from upstream.
Instead, creates one trivial .asd file for each contrib except asdf itself (so that it can be upgraded the asdf way).

Many tweaks to the contribs (mainly their .asd files, but not exclusively), so they build cleanly in this setup. Notably make sb-cover play nicely with logical-pathnames, and sb-introspect/tests DTRT with forcing the test file to be compile every time (so we can introspect the effects to (eval-when (:compile-toplevel) ...).

Also, moves all intermediate output from contrib build from contrib/ to obj/asdf-cache/

Also, creates a obj/sbcl-home/ hierarchy that is used as the $SBCL_HOME for run-sbcl.sh. Ideally, sbcl.core would be moved there, too, instead of being put in output/, and that would simplify some code.

Also, updated ASDF to 2.30.8, which includes some tweaks to make it all work.

[Faré (fahree)]

NB: This also fixes bug #393309
https://bugs.launchpad.net/sbcl/+bug/393309

[Faré (fahree)]

Same patch, with new ASDF 2.31 release rather than development version 2.30.8.

[Faré (fahree)]

Here is an update with ASDF 2.32, and some slight improvements to take advantage of its new working require-system class.

[Faré (fahree)]

And a revised version of the previous patch with a typo fixed and an obsolete comment removed.

[Faré (fahree)]

And here is an update to my patch which takes care of a function having been renamed in asdf.

[Faré (fahree)]

New patch for ASDF 3.0.1 (works better with quicklisp).

[James Y Knight (foom)]

Patch works for me. It also fixes some annoying buildsystem/timestamp issues I ran into with the existing contrib mechanism.

In the existing setup, the contribs require the .lisp source file to be installed next to the fasl for the contrib to load. However, if it so happens that the lisp file gets a later timestamp, asdf will try to rebuild. But, that'll fail, because attempting to run the C compiler to re-grovel fails, as the environment is no longer setup properly (being set by the Makefile/asdf-module.mk).

This new contribs system completely eliminates the possibility of any broken post-sbcl-build rebuild attempts, by totally eliminating the need to ship the lisp files (and in fact doesn't require asdf to load the contribs at all anymore!)

[Faré (fahree)]

Update of the patch, using ASDF 3.0.2.

Now is a good time to push that to SBCL, isn't it? I mean, ASDF has been stable for months.

[Faré (fahree)]

Patch update for the latest SBCL.

[Juho Snellman (jsnell+bugs)]

Looks nice. And conflicts so badly with my parallel contrib build patch that committing this first seems like the decent thing to do :-)

What's up with the DEPENDENCIES Makefile variable in asdf-install and asdf-module.mk? I'm assuming it was relevant at some earlier point of this patch, but is now vestigial. Is that correct, or is it actually still relevant somehow?

[Faré (fahree)]

It was already commented out, I removed the comment.

asdf-install itself is vestigial. Go quicklisp!

Are you trying to build contribs in parallel? Just use POIU...

[Juho Snellman (jsnell+bugs)]

Either I can't read a diff or we're talking of different things :-) I mean the following changes, of introducing a commented out variable and then introducing a use of it:

--- a/contrib/asdf-install/Makefile
+++ b/contrib/asdf-install/Makefile
@@ -1,2 +1,3 @@
 SYSTEM=asdf-install
+#DEPENDENCIES=sb-bsd-sockets sb-posix

--- a/contrib/asdf-module.mk
+++ b/contrib/asdf-module.mk
@@ -27,18 +29,21 @@ endif
- $(MAKE) -C ../asdf
+ for i in asdf $(DEPENDENCIES) ; do $(MAKE) -C ../$$i ; done

Which both look like no-ops to me. Not a problem to revert these, but I wanted to know if this was intentional.

[Faré (fahree)]

They probably should go, indeed, though I don't remember how dependencies are done, in the end.
I probably just let ASDF sort it out in its asdf-cache.

[Faré (fahree)]

Updated patch without vestigial DEPENDENCIES variable.

[Faré (fahree)]

Gah, previous patch removed the make -C ../asdf
Adding it back.

[Faré (fahree)]

Thanks a lot for committing the patch!

Is your parallel something patch related to https://bugs.launchpad.net/sbcl/+bug/1010646 ?