wrong environment for macrolet macroexpansion function in walker

[Christophe Rhodes]

 affects sbcl
 tag walker macrolet
 done

the walker's walk-macrolet walks the macroexpansion function body in the
null environment. That's wrong: CLHS MACROLET says

  The macro-expansion functions defined by macrolet are defined in the
  lexical environment in which the macrolet form appears. Declarations
  and macrolet and symbol-macrolet definitions affect the local macro
  definitions in a macrolet, but the consequences are undefined if the
  local macro definitions reference any local variable or function
  bindings that are visible in that lexical environment.

The simple fix is probably just to use old-env instead of nil in
walk-macrolet walker-environment-bind; that will get some answers wrong
if there are references to local variables or functions, but the
consequences of that case are explicitly unspecified anyway.

However. Trying to construct a test case suggests that maybe I don't
understand enough. I tried something like

    (symbol-macrolet ((x 3))
      (locally (declare (special x))
        (macrolet ((z () x))
          (z))))

expecting to get an error (per "declarations ... affect the local macro
definitions") but instead the main SBCL evaluator, not even the walker
any more, gave me 3. So I'm not marking this bug as confirmed because I
don't understand it well enough.

[Douglas Katzman (dougk)]

 I explored this case when fixing some bugs in the Google EVAL.
 The question is what it means "not to see" a thing that you are not supposed to look at, and unsurprisingly, Lisp implementations
 differ on this.
But (LOCALLY (DECLARE SPECIAL)) is the strangest of the strange cases.
Start with one that's not quite as strange-

(defun f ()
  (symbol-macrolet ((x ''foo))
    (let ((x 3))
      (macrolet ((z () x))
        (z)))))

In this example, the macro Z is using X in its own code and it sees, but should not see, a lexical variable of that name shadowing a lexical macro. SBCL's compiler does something different from other Lisps: we "hop over" the binding that you should not see, and we see the binding that you are permitted to see.
Run that through the compiler, and (F) => FOO.

But try it in brand X lisp and then brand Y lisp:

*** - Invalid access to the value of the lexical variable X from within a MACROLET definition
The following restarts are available:
ABORT :R1 Abort main loop

> Error: While compiling Z :
> Illegal reference to lexically defined variable X., in process listener(1).
> Type :GO to continue, :POP to abort, :R for a list of available restarts.
> If continued: continue compilation ignoring this form

In each case they've said here is something that we *know* you should not see, and "no" we're not going to look past that to see the thing that would be visible. Now suppose X were a bound special - I think the argument is the same - it should be an error.

So back to the strangest of the strange - a free special makes the macro know that there is something which is neither prohibited to see nor expressly permitted to see.
In this case, brand X lisp still tells you that it is illegal to see, and brand Y lisp tells you ' unbound variable '