"The value NIL is not of type SB-C::FUNCTIONAL", DELETE-OPTIONAL-DISPATCH with recursive local function.

[Alastair Bridgewater]

Much to my surprise, the following code:

(defun m-a-o (fun start)
  (let ((current start))
    (labels ((maybe-map (obj obj-tag n-obj-bytes &optional ok)
               (declare (ignore obj obj-tag n-obj-bytes ok)))
             (map-objects-in-range ()
               (multiple-value-bind
                     (object type-code size)
                   (r-o current)
                 (maybe-map object type-code size))
               (map-objects-in-range)))
      (declare (inline maybe-map map-objects-in-range))
      (map-objects-in-range))))

Yields the following when compiled (SBCL 1.1.7.25 or so):

debugger invoked on a TYPE-ERROR in thread
#<THREAD "main thread" RUNNING {AB37001}>:
  The value NIL is not of type SB-C::FUNCTIONAL.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-C::DELETE-OPTIONAL-DISPATCH #<SB-C::OPTIONAL-DISPATCH :%SOURCE-NAME SB-C::.ANONYMOUS. :%DEBUG-NAME (LAMBDA (&OPTIONAL (OBJECT) (TYPE-CODE) (SIZE) &REST #1=#:G13) :IN M-A-O) :TYPE #<SB-KERNEL:BUILT-IN-CLASSOID FUNCTION (read-only)> :WHERE-FROM :DEFINED :ARGLIST (#<SB-C::LAMBDA-VAR :%SOURCE-NAME OBJECT :ARG-INFO #<SB-C::ARG-INFO :KIND :OPTIONAL {AB57B61}> {AB57B01}> #<SB-C::LAMBDA-VAR :%SOURCE-NAME TYPE-CODE :ARG-INFO #<SB-C::ARG-INFO :KIND :OPTIONAL {AB57BF1}> {AB57B91}> #<SB-C::LAMBDA-VAR :%SOURCE-NAME SIZE :ARG-INFO #<SB-C::ARG-INFO :KIND :OPTIONAL {AB57C81}> {AB57C21}> #<SB-C::LAMBDA-VAR :%SOURCE-NAME #1# :FLAGS (IGNORE) :ARG-INFO #<SB-C::ARG-INFO :KIND :REST {AB57D11}> {AB57CB1}>) :ALLOWP NIL :KEYP NIL :MIN-ARGS 0 :MAX-ARGS 0 :MAIN-ENTRY NIL {AB57D79}>)

Removing the recursive call to map-objects-in-range, removing the inline declaration for map-objects-in-range, or removing the &optional argument to maybe-map makes the error go away. Removing the inline declaration for maybe-map doesn't.

I have enough else on my queue right now that trying to dig through the IR1 magic involved here isn't a high enough priority for me to do more than produce this reduced test case. Hopefully someone else can figure this one out, or maybe I'll get back to it at some point this decade.

[Stas Boukarev (stassats)]

A simplified test-case:
(defun bug ()
  (labels ((called (&optional a))
           (recursed (&optional b)
             (called)
             (recursed)))
    (declare (inline recursed called))
    (recursed)))

and it looks like it's related to the inlined recursion, if I do (setf sb-ext:*inline-expansion-limit* 5), it doesn't happen, though it does with 6. So, looks like it breaks something when trying to delete the optional repeatedly.

[Stas Boukarev (stassats)]

I traced the problem down to ir1-convert-hairy-lambda being interrupted by locall-already-let-converted being thrown, which causes a partially initialized optional-dispatch to be left in component-new-functionals of *current-component*. A fix which works for me is to not put the new optional-dispatch there until the function which can throw this tag returns. It works, survives the build and tests, and makes sense, but I'm still uncomfortable with it, since I don't understand why that tag is being thrown in the first place.

[Stas Boukarev (stassats)]

In ae026fe85fef157ff42d6655e5f5e4eef74709f1.