(setf *backend-page-bytes* 262144) breaks build

[Nikodemus Siivola]

Dimitar Kenanov via lists.sourceforge.net to sbcl-bugs:

Hi,
firstly i am working on 64 bit Slackware linux with 64Gb RAM. Version of
SBCL is 1.0.57. Page size of my system is 4096.

I do not know how to program in SBCL but I have a program that uses SBCL
and after a while SBCL exits with the known error:

------------
fatal error encountered in SBCL pid 10894(tid 140737353926400):
An mprotect call failed with ENOMEM. This probably means that the
maximum amount
of separate memory mappings was exceeded. To fix the problem, either
increase
the maximum with e.g. 'echo 262144 > /proc/sys/vm/max_map_count' or
recompile
SBCL with a larger value for GENCGC-CARD-BYTES in
'src/compiler/target/backend-parms.lisp'.
------------

So i went to the specified file and changed the value there to:(setf
*backend-page-bytes* 2097152)) and tried to compile. But then the
compiler exits with the following error:

-----------
obj/from-xc/src/pcl/walk.lisp-obj
[building initial core file in "output/cold-sbcl.core":
writing 2097152 bytes [1 page] from #<SB!FASL::GSPACE :READ-ONLY>
unhandled SB-INT:INVALID-ARRAY-INDEX-ERROR in thread #<SB-THREAD:THREAD
                                                       "main thread"
RUNNING
                                                        {10029612A3}>:
  Index 1 out of bounds for (SIMPLE-VECTOR 1), should be nonnegative
and <1.

0: (SB-DEBUG::MAP-BACKTRACE
    #<CLOSURE (LAMBDA # :IN SB-DEBUG:BACKTRACE) {10664CC54B}>
    :START
    0
    :COUNT
    128)
1: (SB-DEBUG:BACKTRACE
    128
    #<SYNONYM-STREAM :SYMBOL SB-SYS:*STDERR* {1000169203}>)
2: (SB-DEBUG::DEBUGGER-DISABLED-HOOK
    #<SB-INT:INVALID-ARRAY-INDEX-ERROR expected-type: (INTEGER 0 (1))
datum: 1>
    #<unavailable argument>)
3: (SB-DEBUG::RUN-HOOK
    SB-EXT:*INVOKE-DEBUGGER-HOOK*
    #<SB-INT:INVALID-ARRAY-INDEX-ERROR expected-type: (INTEGER 0 (1))
datum: 1>)
4: (INVOKE-DEBUGGER
    #<SB-INT:INVALID-ARRAY-INDEX-ERROR expected-type: (INTEGER 0 (1))
datum: 1>)
5: (ERROR
    SB-INT:INVALID-ARRAY-INDEX-ERROR
    :ARRAY
    #(#(45 6 0 0 0 ...))
    :AXIS
    NIL
    :DATUM
    1
    :EXPECTED-TYPE
    (INTEGER 0 (1)))
6: (SB-INT:INVALID-ARRAY-INDEX-ERROR #(#(45 6 0 0 0 ...)) 1 1 NIL)
7: (SB-KERNEL::INVALID-ARRAY-INDEX-ERROR-HANDLER
    #<unavailable argument>
    #.(SB-SYS:INT-SAP #X7FFFF696F170)
    #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP #X7FFFF696EB80 :TYPE (*

(SB-ALIEN:STRUCT

SB-VM::OS-CONTEXT-T-STRUCT))>
    (213 404 84))
8: (SB-KERNEL:INTERNAL-ERROR
    #.(SB-SYS:INT-SAP #X7FFFF696EB80)
    #<unavailable argument>)
9: ("foreign function: call_into_lisp")
10: ("foreign function: funcall2")
11: ("foreign function: interrupt_internal_error")
12: ("foreign function: #x41458A")
13: (SB!FASL::BVREF
     #S(SB!FASL::BIGVEC :OUTER-VECTOR #(#(45 6 0 0 0 ...)))
     65536)
14: (SB!FASL::WRITE-BIGVEC-AS-SEQUENCE
     #S(SB!FASL::BIGVEC :OUTER-VECTOR #(#(45 6 0 0 0 ...)))
     #<SB-SYS:FD-STREAM
       for "file /home/domine/Downloads/sbcl-1.0.57/output/cold-sbcl.core"
       {10664C1083}>
     :START
     0
     :END
     2097152)
15: (SB!FASL::OUTPUT-GSPACE #<SB!FASL::GSPACE :READ-ONLY>)
16: (SB!FASL::WRITE-INITIAL-CORE-FILE "output/cold-sbcl.core")
17: (SB!VM:GENESIS
     :OBJECT-FILE-NAMES
     (#P"obj/from-xc/src/code/show.lisp-obj"
      #P"obj/from-xc/src/code/early-source-location.lisp-obj"
      #P"obj/from-xc/src/code/early-constants.lisp-obj"
      #P"obj/from-xc/src/code/backq.lisp-obj"
      #P"obj/from-xc/src/code/globals.lisp-obj" ...)
     :SYMBOL-TABLE-FILE-NAME
     "src/runtime/sbcl.nm"
     :CORE-FILE-NAME
     "output/cold-sbcl.core"
     :MAP-FILE-NAME
     "output/cold-sbcl.map"
     :C-HEADER-DIR-NAME
     "output/genesis-2")
18: (SB-INT:SIMPLE-EVAL-IN-LEXENV
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...)
     #<NULL-LEXENV>)
19: (EVAL
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...))
20: (SB-EXT:INTERACTIVE-EVAL
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...)
     :EVAL
     NIL)
21: (SB-IMPL::REPL-FUN NIL)
22: ((LAMBDA () :IN SB-IMPL::TOPLEVEL-REPL))
23: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX
     #<CLOSURE (LAMBDA # :IN SB-IMPL::TOPLEVEL-REPL) {100297060B}>)
24: (SB-IMPL::TOPLEVEL-REPL NIL)
25: (SB-IMPL::TOPLEVEL-INIT)
26: ((FLET #:WITHOUT-INTERRUPTS-BODY-236908 :IN SB-EXT:SAVE-LISP-AND-DIE))
27: ((LABELS SB-IMPL::RESTART-LISP :IN SB-EXT:SAVE-LISP-AND-DIE))

unhandled condition in --disable-debugger mode, quitting
------------

I tried many values but compiling fails with the error above. Compiling
was only success with the default value and 65536. But even with 65536
SBCL exits with the ENOMEM error when i use the program which i have.
Unfortunately its for my work and i cant go around.

How can i solve that problem with the compiling?
Any help or hint will be appreciated!

Thank you
Dimitar

[Paul Khuong (pvk)]

The code to write big vectors to a stream only works by accident. The patch below fixes that issue, but there are other problems, including at least that of finding a memory map for static and read-only spaces.

[Martin Cracauer (cracauer)]

I can confirm that I had gotten the same backtrace as the OP and that the patch fixes that.

Here's the next backtrace in line.

[...]
obj/from-xc/src/code/late-defbangmethod.lisp-obj
obj/from-xc/src/pcl/walk.lisp-obj
[building initial core file in "output/cold-sbcl.core":
writing 2097152 bytes [1 page] from #<SB!FASL::GSPACE :READ-ONLY>
writing 2097152 bytes [1 page] from #<SB!FASL::GSPACE :STATIC>
failed AVER:
(ZEROP REM)
unhandled SB!INT:BUG in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                  {10029A9D33}>:
    failed AVER: (ZEROP REM)
  If you see this and are an SBCL developer, then it is probable that you have
  made a change to the system that has broken the ability for SBCL to compile,
  usually by removing an assumed invariant of the system, but sometimes by
  making an averrance that is violated (check your code!). If you are a user,
  please submit a bug report to the developers' mailing list, details of which
  can be found at <http://sbcl.sourceforge.net/>.

0: (SB-DEBUG::MAP-BACKTRACE
    #<CLOSURE (LAMBDA # :IN SB-DEBUG:BACKTRACE) {10159E91FB}>
    :START
    0
    :COUNT
    128)
1: (SB-DEBUG:BACKTRACE
    128
    #<SYNONYM-STREAM :SYMBOL SB-SYS:*STDERR* {1000169E83}>)
2: (SB-DEBUG::DEBUGGER-DISABLED-HOOK
    #<SB!INT:BUG "~@<failed AVER: ~2I~_~A~:>" {10159E09E3}>
    #<unavailable argument>)
3: (SB-DEBUG::RUN-HOOK
    SB-EXT:*INVOKE-DEBUGGER-HOOK*
    #<SB!INT:BUG "~@<failed AVER: ~2I~_~A~:>" {10159E09E3}>)
4: (INVOKE-DEBUGGER #<SB!INT:BUG "~@<failed AVER: ~2I~_~A~:>" {10159E09E3}>)
5: (ERROR
    SB!INT:BUG
    :FORMAT-CONTROL
    "~@<failed AVER: ~2I~_~A~:>"
    :FORMAT-ARGUMENTS
    ((ZEROP REM)))
6: (SB!INT:BUG "~@<failed AVER: ~2I~_~A~:>" (ZEROP REM))
7: (SB!FASL::OUTPUT-GSPACE #<SB!FASL::GSPACE :STATIC>)
8: (SB!FASL::WRITE-INITIAL-CORE-FILE "output/cold-sbcl.core")
9: (SB!VM:GENESIS
    :OBJECT-FILE-NAMES
    (#P"obj/from-xc/src/code/show.lisp-obj"
     #P"obj/from-xc/src/code/early-source-location.lisp-obj"
     #P"obj/from-xc/src/code/early-constants.lisp-obj"
     #P"obj/from-xc/src/code/backq.lisp-obj"
     #P"obj/from-xc/src/code/globals.lisp-obj" ...)
    :SYMBOL-TABLE-FILE-NAME
    "src/runtime/sbcl.nm"
    :CORE-FILE-NAME
    "output/cold-sbcl.core"
    :MAP-FILE-NAME
    "output/cold-sbcl.map"
    :C-HEADER-DIR-NAME
    "output/genesis-2")
10: (SB-INT:SIMPLE-EVAL-IN-LEXENV
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...)
     #<NULL-LEXENV>)
11: (EVAL
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...))
12: (SB-EXT:INTERACTIVE-EVAL
     (SB!VM:GENESIS :OBJECT-FILE-NAMES *TARGET-OBJECT-FILE-NAMES*
                    :C-HEADER-DIR-NAME "output/genesis-2" ...)
     :EVAL
     NIL)
13: (SB-IMPL::REPL-FUN NIL)
14: ((LAMBDA () :IN SB-IMPL::TOPLEVEL-REPL))
15: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX
     #<CLOSURE (LAMBDA # :IN SB-IMPL::TOPLEVEL-REPL) {10029B060B}>)
16: (SB-IMPL::TOPLEVEL-REPL NIL)
17: (SB-IMPL::TOPLEVEL-INIT)
18: ((FLET #:WITHOUT-INTERRUPTS-BODY-237846 :...

[Douglas Katzman (dougk)]

Paul's patch was applied in 2013, and whatever other issues there were appear fixed.

./run-sbcl.sh --noinform --eval '(format t "~d~%" sb-vm:gencgc-card-bytes)' --quit
(running SBCL from: .)
262144