Failed to compile working code without declareing some functions notinline.

[Anton]

SBCL version 1.2.4, using optima pattern matching library: https://github.com/m2ym/optima

Compiling file with this code will fail without declaring signal-error notinline.

(defun signal-error ()
  (error 'error))

(defun will-fail ()
  (optima:match 1
    ((not 2)
     (signal-error))))

Error report:

; compiling (DEFUN SIGNAL-ERROR ...)
; compiling (DEFUN WILL-FAIL ...)
; file: /home/freeman/err.lisp
; in: DEFUN WILL-FAIL
; (OPTIMA:MATCH 1
; ((NOT 2) (COMMON-LISP::SIGNAL-ERROR)))
; --> LET OPTIMA::%OR BLOCK SB-C::%COMPILE-TIME-TYPE-ERROR
; ==>
; '((BLOCK #:BLOCK2
; (TAGBODY
; (RETURN-FROM #:BLOCK2
; (SYMBOL-MACROLET #
; #))
; #:FAIL3
; (RETURN-FROM #:BLOCK2 NIL)))
; (COMMON-LISP::SIGNAL-ERROR))
;
; caught ERROR:
; don't know how to dump 2 (default MAKE-LOAD-FORM method called).

; --> LET OPTIMA::%OR BLOCK
; ==>
; (SB-C::%COMPILE-TIME-TYPE-ERROR 'SB-C::DUMMY 'NIL '*
; '((BLOCK #:BLOCK2
; (TAGBODY
; (RETURN-FROM #:BLOCK2 #)
; #:FAIL3
; (RETURN-FROM #:BLOCK2 NIL)))
; (COMMON-LISP::SIGNAL-ERROR)))
;
; note: The fourth argument never returns a value.
Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                  {1002BF7023}>:
  The value NIL is not of type (AND ATOM (NOT NULL)).

After declaring singnal-error as notinline or changing ERROR to SIGNAL it'll compile ok.

[Douglas Katzman (dougk)]

This is partially the fault of optima and partially the failure of the compiler to stop processing when things go wrong when attempting to dump a constant object.

The root cause is:
caught ERROR:
; don't know how to dump 2 (default MAKE-LOAD-FORM method called).
This "2" is not the literal 2, but an OPTIMA.CORE:CONSTANT-PATTERN

If you put this in either your source code or the Optima source code, it fixes the problem:
(eval-when (:compile-toplevel)
 (defmethod make-load-form ((self constant-pattern) &optional env)
            (declare (ignore env))
            (make-load-form-saving-slots self)))

As to why declaring the error signaling inline "fixes" the problem, it's because on account of the compiler knowing that ERROR does not return, there becomes some unreachable code, and that is the very same code which couldn't be dumped properly.

[Tomohiro Matsuyama (m2ym-pub)]

Hi, I am the author of optima.

Could you please tell me why make-load-form is called since SBCL 1.2.4? The constant pattern 2 will be disappeared after macroexpand in my understand. Am I missing some points?

[Douglas Katzman (dougk)]

Well, good question.
It turns out that this is probably two different bugs in the compiler, one of which is allegedly a feature.

The feature is that when certain compile-time errors are detected [bear with me], the compiler emits code which at runtime will report the offending s-expression as it appeared in source form.
A quick example:
* (defun bad (a b) (declare (string a) (integer b)) (+ a b))
; in: DEFUN BAD
; (+ A B)
; caught WARNING:
; Derived type of A is (VALUES STRING &OPTIONAL), conflicting with its asserted type NUMBER.

Now if BAD is called with arguments that satisfy the declared types, but violate the ftype of '+' we see a pretty message:
* (bad "hi" 3)
debugger invoked on a SIMPLE-TYPE-ERROR in thread
#<THREAD "main thread" RUNNING {1002C16A63}>:
  Value of A in (+ A B) is "hi", not a NUMBER.

How did assembly code know that it was adding A and B? Because the original form appears as a literal constant.
This is the disassembly, with lots of preceding stuff elided for brevity:
; 121: 488B0518FFFFFF MOV RAX, [RIP-232] ; '((+ A B) A)
; 128: 488943F0 MOV [RBX-16], RAX
; 12C: 488B0515FFFFFF MOV RAX, [RIP-235] ; #<FDEFINITION for SB-C::%COMPILE-TIME-TYPE-ERROR>

This means that the source had to be dumpable, which is simply naive of the compiler to assume. It should be able to at least say that it does not have access to the source (if it doesn't).
This behavior was added in commit a1a34a500b880ab761291350300d8d3184574183 saying
" 1.0.48.9: better source information for compile-time type errors

      Compile-time warning: in addition to the context, also tell exactly
      which form produces the value that is not of the expected type.

      Run-time error: include both the error context and exact form in the
      error message."

As to the error that causes the compiler to want to dump source code at all ... that's the deeper bug.
It was masked by the reported error about inability to dump an s-expression.
This will take further investigation to figure out why it thinks there is actually something wrong with the code to begin with.

[Tomohiro Matsuyama (m2ym-pub)]

Thanks for the thorough explanation. I completely understand the situation. I will fix the problem at optima side by not revealing pattern objects after macro expansion.

[Douglas Katzman (dougk)]

But you shouldn't be required to change anything - this is genuinely a problem in the compiler.
Small example without using optima:

foo.lisp
========
(eval-when (:compile-toplevel :load-toplevel :execute)
  (defstruct silly))
(defmacro macro1 (a b) `(+ (block foo (return-from foo ,a) (frob ,(make-silly))) ,b))
(defmacro frob (arg) `',(type-of arg))
(defun foo (x y)
  (declare (string x) (integer y))
  (macro1 x y))
========

* (compile-file "/tmp/foo.lisp")
; caught ERROR:
; don't know how to dump #S(SILLY) (default MAKE-LOAD-FORM method called).
debugger invoked on a TYPE-ERROR:
  The value NIL is not of type (AND ATOM (NOT NULL)).

It could be argued that FROB must be macroexpanded before dumping the form starting with (+ ...) that contains the type-error. In this case, FROB has a perfectly reasonable expansion:
* (frob #s(silly)) => SILLY

However, in this problem as originally reported, I don't think there was a type-error at all. The code runs fine.

[Tomohiro Matsuyama (m2ym-pub)]

Aside from this problem, I felt there is another problem revealing internal pattern objects during macro expansion. Now, I have fixed it by not revealing them. Thank you again.