Macro expansion to (include ...) not working to export bindings inside a library

[Derick Eddington]

I'm working on porting SRFIs and I made an include/resolve which searches the library-path so the original reference implementation files can be included into the library, and I ran into the below problem. Macro expanding to include in the top-level interaction environment works and makes the bindings available, but when it's done in a library the bindings aren't able to be exported. It doesn't seem to be a problem with (begin ...) splicing; maybe the problem has something to do with include being a built-in macro?

$ cat to-include.scm
(define x 123)
(define y 54321)
(define (f z) (+ x y z))
(define-syntax s (syntax-rules () [(_ a) '(a a a)]))
$
$ ikarus
Ikarus Scheme version 0.0.2patched+ (revision 1181, build 2007-12-03)
Copyright (c) 2006-2007 Abdulaziz Ghuloum

> (library (include-it)
    (export include-it)
    (import (ikarus))
    (define-syntax include-it
      (lambda (stx)
        (syntax-case stx ()
          [(_) #'(include "to-include.scm")]))))
>
> (library (hmm)
    (export x y f s)
    (import (rnrs) (include-it))
    (include-it))
Unhandled exception
 Condition components:
   1. &error
   2. &who: expander
   3. &message: "cannot export unbound identifier"
   4. &irritants: (s)
>

[Derick Eddington (derick-eddington)]

Okay, it has nothing to do with a macro expanding to include. I should have tried this first:

$ cat to-include.scm
(define x 123)
(define y 54321)
(define (f z) (+ x y z))
(define-syntax s (syntax-rules () [(_ a) '(a a a)]))
$ ikarus
Ikarus Scheme version 0.0.2patched+ (revision 1181, build 2007-12-03)
Copyright (c) 2006-2007 Abdulaziz Ghuloum

> (library (hmm)
    (export (x y f s))
    (import (rnrs) (only (ikarus) include))
    (include "to-include.scm"))
Unhandled exception
 Condition components:
   1. &error
   2. &who: expander
   3. &message: "invalid export"
   4. &irritants: ((x y f s))
>

[Abdulaziz Ghuloum (aghuloum)]

On Dec 4, 2007, at 10:16 AM, Derick Eddington wrote:

> (export (x y f s))

Extra parenthesis: (export x y f s)

Try harder, you'll get it :-)

[Derick Eddington (derick-eddington)]

Confusing myself by rushing :-/

But then, see my example in the first post. So then, using include directly works to export the included bindings, but a macro which expands to include is not working.

[Derick Eddington (derick-eddington)]

I'm guessing this is happening because the expander/library stuff isn't seeing the included bindings before it checks if the exported bindings exist, or there's some mismatch with the syntax-object contextual info, or probably there's some deeper reason which is beyond me why it is correct; but if it could be fixed/done, it seems very useful to be able to have macros which can expand to include.

[Abdulaziz Ghuloum (aghuloum)]

On Dec 4, 2007, at 11:34 AM, Derick Eddington wrote:

> [...] or there's some mismatch with the syntax-object contextual info,

Right. I'll try to explain it.

The include macro is non-hygienic. It introduces code into the
context in which it was called. So, if you have a file called foo.ss
containing

;;; this is foo.ss

(define foo 12)

;;; end of foo.ss

then you can do the following
(let ()
   (include "foo.ss")
   (+ foo foo))

and this works. Why it works is because include first reads the file
(obtaining a list of s-expressions), then performs a non-hygienic
datum->syntax to add the contextual information. Now datum->syntax
takes an identifier and a datum, where the identifier contains the
actual contextual information. Because the file name is not an
identifier (it's a string), it has no contextual information. So,
include would have to use the first word (i.e. the include keyword)
as the identifier. Include may be defined as:

(define-syntax include
   (lambda (x)
     (syntax-case x ()
       [(ctxt name)
        (cons #'begin
          (with-input-from-file
            (syntax->datum #'name)
            (lambda ()
              (let f ()
                (let ([x (read)])
                  (cond
                    [(eof-object? x) #'()]
                    [else
                     (cons
                       (datum->syntax #'ctxt x)
                       (f))]))))))])))

Now you can write your include-it macro in a similar way, changing
only the part where the file name is determined. And that would make
two broken variations of include.

One solution is to make the name part of the include an identifier
instead of a string. So, you would do (include foo.ss) instead of
(include "foo.ss"). This would kind of work because the context
would be extracted from the identifier foo.ss and not from the
keyword include.

Another option is to define a separate "include-into" macro like:

(define-syntax include-into
   (lambda (x)
     (syntax-case x ()
       [(_ ctxt name)
        --- same code as above ---])))

And define include as:

(define-syntax include
   (lambda (x)
     (syntax-case x ()
       [(ctxt name)
        (include-into ctxt name)])))

Now your code can use "include-into", passing an appropriate context,
and it would work.

BTW: this problem is not limited to include but exists in all other
non-hygienic macros like define-record-type and import. We can
sometimes fake it (like I did in bug 162741). I think the include-
into is a better solution. So, will add.

Aziz,,,

[Abdulaziz Ghuloum (aghuloum)]

Fix to:

(define-syntax include
   (lambda (x)
     (syntax-case x ()
       [(ctxt name)
        #'(include-into ctxt name)])))

[Derick Eddington (derick-eddington)]

Thank you very much for the explanation and for advancing my macro understanding, and for making a solution!

[Abdulaziz Ghuloum (aghuloum)]

Added "include-into" in revision 1187.

[Abdulaziz Ghuloum (aghuloum)]

This bug report is about to be closed as the fix comitted
previously will be incorporated in the next 0.0.3 release of
Ikarus Scheme, scheduled for January 31, 2008. A release
candidate tarball is available for download from:
http://www.cs.indiana.edu/~aghuloum/ikarus/ikarus-0.0.3-rc1.tar.gz
Please do test it if you have the time and report any issues
you might encounter. Thank you very much for your support.
(Sorry for the duplicates; I'm updating every open bug.)