nconc gc and heap exhausted

[il71]

# Linux amd64

$ sbcl --version
SBCL 2.4.3

$ cat nconc-gc.lisp

(let ((counter 0))
  (defun get-data ()
    (mapcar #'digit-char-p (coerce (princ-to-string (incf counter)) 'list))))

;; If nconc is used this leads to heap exhausted rather quickly
;; append is ok

(defun scan (word times)
  (let ((len-word (length word))
        (pos 1)
        (max 0)
        (buffer nil))
    (labels
        ((do-match ()
           (tagbody
             begin
             (loop while (< (length buffer) len-word) do (fetch))
             (when (loop for x in word
                         for y in buffer
                         always (= x y))
               (when (zerop (decf times))
                 (return-from scan pos)))
             (drop)
             (go begin)))
         (drop ()
           (incf pos)
           (pop buffer))
         (fetch ()
           (setf buffer (nconc buffer (get-data)))

           ;; buffer is never big
           (when (> (length buffer) max)
             (setf max (length buffer))
             (print max)
             (finish-output))))

      (do-match))))

(print (scan (list 1 1 2 3 4 5) (expt 10 5)))

$ sbcl --dynamic-space-size 8Gb --quit --load nconc-gc.lisp

# Heap exhausted during garbage collection

[Douglas Katzman (dougk)]

The fixes are either:
1) add --with-mark-region-gc to your build and/or wait for that to become the default
OR
2) (SETF (GENERATION-NUMBER-OF-GCS-BEFORE-PROMOTION 0) #x7fffffff)

In either option you don't need the 8GB dynamic space.

I don't think it's worth putting any time into improving the now-quarter-century-old default garbage collector for the particular case of a clever little algorithm that outsmarts the default set of heuristics on when to collect more aggressively.
As to the default config and GC parameters, this is a "won't fix" since it just comes down to tuning.