dynamic-extent and &rest failure

[Stas Boukarev]

(apply (lambda (&rest rest) (declare (dynamic-extent rest)) (reduce #'+ rest)) (make-list 100000 :initial-element 1))

ends up with

CORRUPTION WARNING in SBCL pid 32097(tid 140737354004208):
Memory fault at f4d25960 (pc=0x1002c38d4a, sp=0x7ffff4d25960)
The integrity of this image is possibly compromised.
Continuing with fingers crossed.

Triggering of that error depends on the length of the list, it seems to be lower on x86-64 than on x86. Increasing it even further will end up in ldb with control stack exhaustion (the same without dynamic-extent).

[Alastair Bridgewater (alastair-bridgewater)]

A quick analysis: one hundred thousand elements times four bytes each (eight bytes on x86-64) is four hundred KB (eight hundred on x86-64). And that's from the APPLY, d-x listifying the rest-arg costs twice as much. You are attempting to blow your stack here (about 2.4 megs of stack usage for one call on x86-64).

The "corruption warning" failure mode is because LISTIFY-REST-ARGS performs a bulk allocation and then fills the space from low addresses to high, while the stack grows downwards on x86oids. This can easily miss the guard page completely (and can do so for as few as 1025 parameters on x86 or 513 on x86-64).

For some reason CALL-ARGUMENTS-LIMIT is set to MOST-POSITIVE-FIXNUM, which is a theoretical limit in some sense, but far beyond the practical limit imposed by the size of the control stack (then again, we have the same issue with array dimensions).

So, given that this is what the situation is, what is the correct behavior for when the user blows his stack in this manner, why, and how do we make it happen?

[Nikodemus Siivola (nikodemus)]

How about we set the CALL-ARGUMENTS-LIMIT to a reasonable number (say 1024), and compute the required stack guard page sizes based on that?

[Gábor Melis (melisgl)]

On Thu, Sep 2, 2010 at 2:21 PM, Nikodemus Siivola
<email address hidden> wrote:
> How about we set the CALL-ARGUMENTS-LIMIT to a reasonable number (say
> 1024), and compute the required stack guard page sizes based on that?

Maybe that's a necessary step, but I can't see how it would be a
general solution. I mean it would blow up the same even if the list is
not coming from &REST, right?

[Douglas Katzman (dougk)]

obviously not "high" importance if open for 11 years