Steel Bank Common Lisp

Saved core executables are "slow" to start up

Reported by Zach Beane on 2010-04-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
SBCL
Wishlist
Unassigned

Bug Description

I created an executable with this file "hello.lisp":

(sb-ext:save-lisp-and-die "hello-world"
                          :executable t
                          :toplevel (lambda ()
                                      (write-line "Hello, world!")))

and "sbcl --no-userinit --no-sysinit --load hello.lisp"

The resulting executable is somewhat slow to execute, with this output representative of several consecutive runs:

$ time ./hello-world
Hello, world!

real 0m0.065s
user 0m0.000s
sys 0m0.070s

By comparison, this "hello-world.pl" script is 10 times faster:

#!/usr/bin/perl
print "Hello world!\n";

$ time ./hello-world.pl
Hello, world.

real 0m0.006s
user 0m0.000s
sys 0m0.000s

It would be nice if saved core executables started up really, really fast.

Changed in sbcl:
status: New → Confirmed
importance: Undecided → Wishlist
Lutz Euler (lutz-euler) wrote :

64-bit SBCL under Linux spends most of its startup time in "gc_init".
This is easy to improve.

I measured the following runtimes for your "Hello world" scripts/images:

Perl: 0.009 s
SBCL 32-Bit: 0.025 s
SBCL 64-Bit: 0.080 s

With the attached patch this improves as follows:

patched 32-Bit: 0.022 s
patched 64-Bit: 0.020 s

Here is the output of "/usr/bin/time ./hello-world" (edited for readability)
for the 64-bit version which shows that the memory footprint is reduced
notedly, too:

Originally:
0:00.07 elapsed
145152 K maxresident
17632 minor pagefaults

With the patch:
0:00.01 elapsed
14768 K maxresident
1295 minor pagefaults

The patch simply removes an unnecessary initialization loop for the array
"page_table_pages" that defeats the operating system's and C library's
efforts to make "calloc" efficient. Please see the comment in the patch
for details.

Please take the patch as preliminary. Given some help, I would like to
improve it:

* As there is a small improvement for 32-bit SBCL, too, not only gencgc
  platforms may be affected. I have not yet looked at "cheneygc.c" but
  presumably that is the only other file in active use containing a
  definition of "gc_init"?
* Without the loop the page structures are correctly initialized only
  as long as FREE_PAGE_FLAG == 0. Thus, for maintainability, an assertion
  or at least a comment at the right place seems sensible.
  If the former: Is there an SBCL way for compile-time assertions in C files?
  If the latter: Any suggestions where a warning should be placed?
* The comment in the patch is too long and I would like to reword it.
  Should these explanations instead be put somewhere else, possibly the
  documentation?
* Presumably, all unixoid operating systems behave the same as Linux here.
  I am curious if that is indeed the case on Solaris, the BSD's and the Mac
  but can't verify this myself.

Kind regards,

Lutz

Lutz Euler (lutz-euler) wrote :
tags: added: review
Nikodemus Siivola (nikodemus) wrote :

Cheneygc should not be affected.

Assertion and comment sounds like a good idea, yes. No, we don't have a specific way for compile-time assertions that I know.

The calloc site should also get a comment that we rely on the zeroing: as long as that and the assert exist, FREE_PAGE_FLAG site may have, but doesn't strictly speaking need a comment.

Maybe leave the loop in there in the comment instead of explaining what used to be there?

Lutz Euler (lutz-euler) wrote :

Finally I found the time to look into this again.

In the meantime the default dynamic space size on 64-bit has been halved so
that the effect of the suggested change is less pronounced. Nevertheless it
still halves the runtime.

Thanks to Nikodemus for the comments. I attach a new version of the patch
that takes these into consideration.

Greetings,

Lutz

Zach Beane (xach) wrote :

Very nice improvement in time! I'd love to see this merged.

Changed in sbcl:
assignee: nobody → Nikodemus Siivola (nikodemus)
status: Confirmed → In Progress
Nikodemus Siivola (nikodemus) wrote :

In 1.0.46.12.

Changed in sbcl:
assignee: Nikodemus Siivola (nikodemus) → nobody
status: In Progress → Fix Committed
Changed in sbcl:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers