Defuns in consequence and alternate parts of the "if" operator get evaluated always

Bug #1758275 reported by Adam Blank on 2018-03-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
SBCL
Low
Unassigned

Bug Description

- Observed in:
SBCL 1.3.1.debian
with:
(:SWANK :SB-BSD-SOCKETS-ADDRINFO :64-BIT :64-BIT-REGISTERS :ALIEN-CALLBACKS
 :ANSI-CL :ASH-RIGHT-VOPS :C-STACK-IS-CONTROL-STACK :COMMON-LISP
 :COMPARE-AND-SWAP-VOPS :COMPLEX-FLOAT-VOPS :CYCLE-COUNTER :ELF :FLOAT-EQL-VOPS
 :FP-AND-PC-STANDARD-SAVE :GENCGC :IEEE-FLOATING-POINT :INLINE-CONSTANTS
 :INTEGER-EQL-VOP :INTERLEAVED-RAW-SLOTS :LARGEFILE :LINKAGE-TABLE :LINUX
 :LITTLE-ENDIAN :MEMORY-BARRIER-VOPS :MULTIPLY-HIGH-VOPS :OS-PROVIDES-BLKSIZE-T
 :OS-PROVIDES-DLADDR :OS-PROVIDES-DLOPEN :OS-PROVIDES-GETPROTOBY-R
 :OS-PROVIDES-POLL :OS-PROVIDES-PUTWC :OS-PROVIDES-SUSECONDS-T
 :PACKAGE-LOCAL-NICKNAMES :PRECISE-ARG-COUNT-ERROR :RAW-INSTANCE-INIT-VOPS
 :SB-AFTER-XC-CORE :SB-CORE-COMPRESSION :SB-DOC :SB-EVAL :SB-FUTEX :SB-LDB
 :SB-PACKAGE-LOCKS :SB-SIMD-PACK :SB-SOURCE-LOCATIONS :SB-TEST :SB-THREAD
 :SB-UNICODE :SB-XREF-FOR-INTERNALS :SBCL :STACK-ALLOCATABLE-CLOSURES
 :STACK-ALLOCATABLE-FIXED-OBJECTS :STACK-ALLOCATABLE-LISTS
 :STACK-ALLOCATABLE-VECTORS :STACK-GROWS-DOWNWARD-NOT-UPWARD :SYMBOL-INFO-VOPS
 :UNIX :UNWIND-TO-FRAME-AND-CALL-VOP :X86-64)
run at:
Linux 4.4.0-116-generic #140~14.04.1-Ubuntu SMP Fri Feb 16 09:25:20 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

- But still present in:
SBCL 1.4.5
with:
(:64-BIT :64-BIT-REGISTERS :ALIEN-CALLBACKS :ANSI-CL :ASH-RIGHT-VOPS
 :C-STACK-IS-CONTROL-STACK :CALL-SYMBOL :COMMON-LISP :COMPACT-INSTANCE-HEADER
 :COMPARE-AND-SWAP-VOPS :COMPLEX-FLOAT-VOPS :CYCLE-COUNTER :ELF :FLOAT-EQL-VOPS
 :FP-AND-PC-STANDARD-SAVE :GCC-TLS :GENCGC :IEEE-FLOATING-POINT :IMMOBILE-CODE
 :IMMOBILE-SPACE :INLINE-CONSTANTS :INTEGER-EQL-VOP :LARGEFILE :LINKAGE-TABLE
 :LINUX :LITTLE-ENDIAN :MEMORY-BARRIER-VOPS :MULTIPLY-HIGH-VOPS
 :OS-PROVIDES-DLADDR :OS-PROVIDES-DLOPEN :OS-PROVIDES-GETPROTOBY-R
 :OS-PROVIDES-POLL :OS-PROVIDES-PUTWC :OS-PROVIDES-SUSECONDS-T
 :PACKAGE-LOCAL-NICKNAMES :RAW-INSTANCE-INIT-VOPS :RAW-SIGNED-WORD
 :RELOCATABLE-HEAP :SB-DOC :SB-EVAL :SB-FUTEX :SB-LDB :SB-PACKAGE-LOCKS
 :SB-SIMD-PACK :SB-SOURCE-LOCATIONS :SB-THREAD :SB-UNICODE :SBCL
 :STACK-ALLOCATABLE-CLOSURES :STACK-ALLOCATABLE-FIXED-OBJECTS
 :STACK-ALLOCATABLE-LISTS :STACK-ALLOCATABLE-VECTORS
 :STACK-GROWS-DOWNWARD-NOT-UPWARD :SYMBOL-INFO-VOPS :UNBIND-N-VOP
 :UNDEFINED-FUN-RESTARTS :UNIX :UNWIND-TO-FRAME-AND-CALL-VOP :X86-64)

- This code shows the problem:
(defun foo (n)
  (if (< n 3)
      (defun bar (n) (format t "bar ~a~%" n)))
  (bar n))

(foo 2)
bar 2
NIL

(foo 10)
bar 10
NIL

- But this one works correctly:
(defun foo (n)
  (if (< n 3)
      (defun bar (n) (format t "bar ~a~%" n))))

(foo 10)

NIL
(bar 1) => ERROR

(foo 1)

BAR
* (bar 1)
bar 1
NIL

-This code also shows the problem:
(defun foo (n)
  (format t "foo ~a~%" n)
  (if (< n 3)
      (defun foo (n) (format t "bar ~a~%" n)))
  (if (> n 1)
      (foo (- n 1))))

(foo 2)
foo 2
WARNING: redefining COMMON-LISP-USER::FOO in DEFUN
bar 1
NIL

(foo 10)
foo 10
bar 9
NIL

- And so does this one:
(defun foo (n)
  (format t "foo ~a~%" n)
  (if (< n 3)
      (defun foo (n) (format t "bar ~a~%" n))
    (defun foo (n) (format t "baz ~a~%" n)))
  (if (> n 1)
      (foo (- n 1))))

(foo 2)
foo 2
WARNING: redefining COMMON-LISP-USER::FOO in DEFUN
baz 1
NIL

(foo 10)
foo 10
WARNING: redefining COMMON-LISP-USER::FOO in DEFUN
baz 9
NIL

Richard M Kreuter (kreuter) wrote :
Download full text (3.9 KiB)

Hello Adam,

Please oblige me if I explain things you already understand; there are a few stylistic aspects of your code samples that suggests that you are/were new to Common Lisp when you wrote this bug report, and so I'm attempting to ``level set'' on the relevant details.

[A] I suspect that you might know this, but for clarity, in Common Lisp, DEFUN has a persistent effect on the global environment (see footnote [1] below). In consequence, your first version of FOO,

(defun foo (n)
  (if (< n 3)
      (defun bar (n) (format t "bar ~a~%" n)))
  (bar n))

has the the following meaning: when FOO is called with an argument, N, that's less than 3, define the function BAR in the global environment; and in all cases, invoke BAR on N.

So when you call FOO with N of 2, BAR gets defined and called; when you subsequently call FOO with N of 10, BAR does not get defined again, but its definition remains from the previous invocation of FOO. So there is a function definition for BAR, and it does get called. This is as you showed in your transcript:

(foo 2)
bar 2
NIL

(foo 10)
bar 10
NIL

[B] However, you say about your second version of FOO that "this one works correctly"

--------
(defun foo (n)
  (if (< n 3)
      (defun bar (n) (format t "bar ~a~%" n))))

(foo 10)

NIL
(bar 1) => ERROR

(foo 1)

BAR
* (bar 1)
bar 1
NIL
--------

That you describe the behavior in this transcript as correct suggests that you're aware that the effect of (DEFUN BAR...) inside FOO persists after FOO returns.

[C] However, the third and fourth code fragments in your bug report indicates that my tentative inference in [B] could be mistaken, though it's hard to say, because the situation is muddied by a separate concern: both of the third and fourth versions of FOO (conditionally) redefine FOO before invoking it recursively. Such code has undefined consequences per the Common Lisp standard section 3.2.2.3 bullet 4:

http://www.lispworks.com/documentation/HyperSpec/Body/03_bbc.htm

What I believe accounts for the results shown in your third and fourth transcripts is that SBCL attempts to inline the recursive call to FOO, and produces an unexpected result -- but, again, since both the third and fourth versions of FOO have undefined consequences, all results are notionally valid. (There are a couple things you can do in SBCL that will cause the third and fourth versions of FOO to behave in accordance with how you might evaluate these calls in your mind's eye -- declaring FOO NOTINLINE before compiling it, or setting SB-EXT:*EVALUATOR-MODE* to :INTERPRET before evaluating the DEFUNs -- but since that code does not have defined consequences, the fact that these happen to "work" is not required, and in principle subject to change.)

[E] Finally, as to the subject line of your bug report, "Defuns in consequence and alternate parts of the "if" operator get evaluated always", I hope that the above analyses clarify that the behavior you're seeing is not a bug in either IF or DEFUN (or some interaction between them), but rather a mix of standardly defined Common Lisp behavior (the first and second fragments) and undefined consequences in Common Lisp (the third and fourth fragments)....

Read more...

Adam Blank (ablank) wrote :

Hello Richard,

thank you for looking into this issue.
I am truly impressed how scrupulous and inquisitive you were about it.

There are many aspects of your comment I'd like to respond to, so please excuse
my possibly chaotic style of writing.

First and foremost, I'd like to express how excited I am to be able to be involved
in any type of technical process regarding my favourite implementation of Common Lisp.
You were right to notice, that I am not a professionally experienced lisper. I've been
learning it and about it for about 1,5 years now, but only in my private time. I've
been working as a low level developer for 9 years and surprisingly I feel in love with
Lisp the moment I found out about it. At the moment of writing the bug report I had read about 6
books and a few articles about it, but had only seen ASDF and SBCL as examples of large Lisp
projects.

Before I get to the point, let me apologise for wasting your time by not making myself clear
in the bug report. You had the full right to misunderstand my examples. I see that I have not once
mentioned, that every subsequent invocation of my unrealistically academic example code had been
run in a fresh and clean session of SBCL (please see my further comments).

Now please allow me to refer to your comments using your original indexing:

[A] Yes, indeed I knew what DEFUN did. If you consider that each invocation of foo took place in
a clean new image, then I suppose things become slightly more interesting:

(defun foo (n)
  (if (< n 3)
      (defun bar (n) (format t "bar ~a~%" n)))
  (bar n))

(foo 10)
bar 10
NIL

If this is correct, then I'd truly appreciate knowing why.

[B] Yes I am aware of that.

[C] Yes indeed I had no knowledge of the undefined behaviour. I guess I'd been putting too much trust
in the simplified description of DEFUN as setting the symbol function of a symbol, to a given
function object. I'd also been under an impression, that since "(...) defun can be used to define a new
function, to install a corrected version of an incorrect definition, to redefine an already-defined function,
or to redefine a macro as a function (...)", this code weird and unpractical as it was, would behave as I expected.

[E] I guess we should return to this one after the discussion is truly over.

[1] You were naturally right to notice how peculiar and stylistically dubious the code examples were,
but let me stress once more, that they were forged specifically to exhibit certain behaviour.

As for the code layout, I use emacs and slime as Lisp development tools, so any suspicious layout
could have only resulted from my manual editing of the bug report.

Looking forward to your response,
Adam

Stas Boukarev (stassats) wrote :

It's not evaluating the defun, it's taking the named-lambda used by the defun and making a local call to it. This is a known issue.

Changed in sbcl:
status: New → Triaged
importance: Undecided → Low
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers