[SBCL 1.2.3] --break-on-failure not taken into account when running distribution tests

[Chris Wagner]

The tests/run-tests.sh runtime option --break-on-error is not fully taken into account. The problem lies in the function run-in-child-sbcl (see in tests/run-tests.lisp) which spawns a new instance of sbcl with an input stream set to /dev/null. I believe that this is inconsistent with the code of run-tests.sh whose help reads:

   Options:

      --break-on-failure Break into the debugger when a test fails
                                                    unexpectedly

The inconsistency comes from run-in-child-sbcl() preventing the user from interacting with the debugger:

;;; Example

$ (cd tests && sh run-tests.sh --break-on-failure compare-and-swap.impure.lisp)
/running tests on '/Users/christianwagner/src/Lisp/archs/sbcl-1.2.3/tests/../src/runtime/sbcl --core /Users/christianwagner/src/Lisp/archs/sbcl-1.2.3/tests/../output/sbcl.core --lose-on-corruption --noinform --no-sysinit --no-userinit --noprint --disable-debugger'

// Running impure tests (#<FUNCTION RUN-TESTS::LOAD-TEST>)
// Running /Users/christianwagner/src/Lisp/archs/sbcl-1.2.3/tests/compare-and-swap.impure.lisp
[...]

::: Running :ATOMIC-PUSH
CORRUPTION WARNING in SBCL pid 21358(tid 140735262114576):
Memory fault at b0095010 (pc=0x7fff8c61ade9, sp=0x11fe9f0)
The integrity of this image is possibly compromised. Note: lose_on_corruption_p=0
Continuing with fingers crossed.
::: UNEXPECTED-FAILURE :ATOMIC-PUSH
    due to #<SB-SYS:MEMORY-FAULT-ERROR {1002E5A473}>:
        "Unhandled memory fault at #xB0095010."

debugger invoked on a SB-SYS:MEMORY-FAULT-ERROR in thread
#<THREAD "main thread" RUNNING {1002C21473}>:
  Unhandled memory fault at #xB0095010.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE ] Continue
  1: [RETRY ] Retry EVAL of current toplevel form.
[...]
0]
WARNING: Starting a select(2) without a timeout while interrupts are disabled.
h
8
fatal error encountered in SBCL pid 21356(tid 140735262114576):
SIGABRT received.

test failed, expected 104 return code, got 1
$ h
-bash: h: command not found
$ 8

;;; I used SIGABRT to stop the test run when I was not able to interact with the debugger.
;;;
;;; End of example

Changing the input specification in tests/run-tests.lisp
from:
   :input #-win32 devnull #+win32 sb-sys:*stdin*
to:
  :input #-win32 t #+win32 sb-sys:*stdin*
solves my problem.

** Platform
- Distribution of SBCL.1.2.3
- Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64
- MacOS/X 10.9.5 ("Mavericks")

[Douglas Katzman (dougk)]

Yes, debugging test failures is hard, especially with '.impure' tests.
This particular one I sort of recognize, and it always had the feeling of a bug in darwin itself iirc.
Anyway there's nothing we can do about it now. Current sbcl on current darwin works.