Problem with immobile allocation of very large code objects

Bug #1823451 reported by Stas Boukarev on 2019-04-06
This bug affects 1 person
Affects Status Importance Assigned to Milestone

Bug Description

(sb-c:define-vop (nop)
  (:info count)
  (:generator 3
              (loop repeat count
                    do (sb-assem:inst nop))))

(defun test ()
  (sb-sys:%primitive nop (expt 2 20)))

The value
is not of type
when binding SB-ALIEN::VALUE

Douglas Katzman (dougk) wrote :

COMPILE-FILE is virtually incapable of producing such a large code object in anything other than contrived example such as this. An organic example might motivate a fix, otherwise it is working as designed.

Stas Boukarev (stassats) wrote :

That error was reported to me, but without any code.

Stas Boukarev (stassats) wrote :

Here it is without any tricks:

(defun foo (x)
  (declare (list x))
  . #. (make-list 100000
                  :initial-element '(setf * (length x))))

Douglas Katzman (dougk) wrote :

Well obviously, I wasn't asking _how_ to make it happen, I was asking _why_.

Supporting deliberately poorly structured lisp is not the intent of immobile code. QPX is one of the most abusive (on the compiler) applications in the wild, and has been running without problem for years, and well within a safety margin of the 1MB limit ("large" functions are about 100KB)
The easy fix is to have this user opt-out of the feature. The other easy fix is to make it opt-in only (which would extremely sad for everyone)
A correct fix would be to allow huge code in the page-crossing map, but I would venture to say that this falls into the realm of "You're holding it wrong" until seeing a bug report by this user.

Stas Boukarev (stassats) wrote :

I don't have the original code, but the fact that somebody has stumbled on that error means that it's a bit suboptimal. I did suggest (setf sb-c::*compile-file-to-memory-space* :dynamic), but that robs of some of the nice features provided by immobile code.

What is the decision time cut-off for immobile/dynamic? Can it be done before assembling? Can we catch the error and re-assemble to dynamic space instead?

Stas Boukarev (stassats) wrote :

Changing the short to int uses two more megabytes for varyobj_page_scan_start_offset and allows 2^24 NOPs to be allocated.

Stas Boukarev (stassats) on 2019-04-07
Changed in sbcl:
status: New → Fix Committed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers