GC and memory-hungry parallel threads
This is (at least) a specific case of "wanting a better documentation on tuning the GC",
When running multiple threads in parallel, SBCL tends to exhaust its heap during GC
even though only a small fraction of the dynamic-space-size is used for live data
at any time. Specifically, in a 64bit VM with a 4Gb dynamic-space-size for SBCL
and 5 parallel worker threads, temporarily allocating 5000 * (2500 * 8 bytes) = 100MB
each time, the live data should not be much more than 500MB total at any time,
which is just 1/8 of the available space. Is this expected?
For the development of a web-application, we need to know how much live data SBCL can safely accomodate.
What is the worst-case memory requirement factor for a given amount of live data?
Is it really 2 * (1+ highest-
Can this ratio be improved, maybe by using fewer generations? How?
Does it make sense to build SBCL for SMP with a non-generational GC? Is it possible?
Could it be a workaround to manually trigger a :full GC at some point when it's still safe?
How to tune the GC for a robust web-application with many worker threads and varying allocations?
Testcase (see attachment):
1. build executable with
2. run e.g. like this:
$ sbcl --version
$ uname -a
Linux leo-2013 2.6.32-
(:ALIEN-CALLBACKS :ANSI-CL :ASH-RIGHT-VOPS :C-STACK-
:FLOAT-EQL-VOPS :GENCGC :IEEE-FLOATING-
:LINKAGE-TABLE :LINUX :LITTLE-ENDIAN :MEMORY-
:SB-DOC :SB-EVAL :SB-FUTEX :SB-LDB :SB-PACKAGE-LOCKS :SB-SIMD-PACK