&rest list type derivation is brittle

[Nikodemus Siivola]

(in-package :sb-c)

(defknown compiler-derived-type (t) (values t t) (unsafe))

(deftransform compiler-derived-type ((x) (t) * :node node)
  (delay-ir1-transform node :optimize)
  (let ((type (type-specifier (lvar-type x))))
    `(values ',type t)))

(defun compiler-derived-type (x)
  (declare (ignore x))
  (values t nil))

(export 'compiler-derived-type)

(in-package :cl-user)

;;; As expected
CL-USER> (funcall (compile nil `(lambda (&rest list)
                                  (sb-c:compiler-derived-type list))))
LIST
T

;;; Oops. We should get LIST, T here too.
CL-USER> (funcall (funcall (compile nil `(lambda ()
                                           (lambda (&rest list)
                                             (sb-c:compiler-derived-type list))))))
T
T

[Nikodemus Siivola (nikodemus)]

In the first case, the VARARGS-ENTRY is turned into a :LET, and PROPAGATE-LET-ARGS gets the type to the LAMBDA-VAR.

In the second case, the VARARGS-ENTRY is not turned into a :LET, and PROPAGATE-LET-ARGS doesn't happen.

[Nikodemus Siivola (nikodemus)]

It isn't turned into a :LET because it remains an :OPTIONAL -- the OPTIONAL-DISPATCH for (LAMBDA (&REST REST-LIST)) never gets deleted in the second case.

[Nikodemus Siivola (nikodemus)]

...and the optional dispatch isn't deleted because the RETURN from the outer lambda holds it alive.

OK. This seems clear so far. However: COMPILE ads a TL-XEP lambda around the first case as well -- why doesn't that keep the optional dispatch alive the same way?

[Nikodemus Siivola (nikodemus)]

Rewind, another vector of attack. In case 1 PROPAGATE-LET-ARGS is reponsible. In case 2 PROPAGATE-LOCAL-CALL-ARGS is used instead, which punts due to the lambda having an OPTIONAL-DISPATCH.

Comment says:

;;; If the function has an XEP, then we don't do anything, since we
;;; won't discover anything.

However, allowing propagation for lambdas with an optional dispatch doesn't seem to hurt, and allows the type to be seen.

[Nikodemus Siivola (nikodemus)]

In 1.0.43.26.

[Kilian Sprotte (kilian-sprotte)]

I noted that the change committed in 1.0.43.26 affects (breaks) DIVIDE-AND-CONQUER-FORCE of screamer.

As a test-case:

(in-package #:screamer-user)

(defun pythagorean-triplesv (n)
  (all-values
    (solution
     (let ((a (an-integer-betweenv 1 n))
    (b (an-integer-betweenv 1 n))
    (c (an-integer-betweenv 1 n)))
       (assert! (=v (+v (*v a a) (*v b b)) (*v c c)))
       (list a b c))
     (reorder #'range-size
       #'(lambda (x) (< x 1e-6))
       #'>
       #'divide-and-conquer-force))))

(pythagorean-triplesv 5) should => ((3 4 5) (4 3 5))

but now it errs with:
The value (1 2) is not of type NULL.
   [Condition of type TYPE-ERROR]

Recompiling DIVIDE-AND-CONQUER-FORCE with the PROPAGATE-LOCAL-CALL-ARGS
of 1.0.43.25 fixes it again.

[Nikodemus Siivola (nikodemus)]

Reduced test case:

(DEFUN USE (X)
  X)

(DEFUN OOPS (VARIABLE)
  (LET ((CONTINUATION
         (LAMBDA
             (&OPTIONAL DUMMY &REST OTHER)
           (DECLARE (IGNORE OTHER))
           (USE DUMMY)
           (USE VARIABLE))))
    (FUNCALL CONTINUATION (LIST 1 2))))

(OOPS T)

The value (1 2) is not of type NULL.
   [Condition of type TYPE-ERROR]

[Nikodemus Siivola (nikodemus)]

Regression fixed in 1.0.43.71, thanks for the heads up!