Threading invariant loss

[Douglas Katzman]

Trying to run an ordinary parallelized build cause a failure in the host lisp.
The host is SBCL 1.5.8 x86-64 linux.

...
; Subprocess 189370 exit status 0
; loading file "/tmp/sbcl/src/code/late-format.lisp":

; wrote /tmp/sbcl/obj/from-xc/src/code/late-format.lisp-obj-scratch
; compilation finished in 0:00:00.020
; Subprocess 189524 exit status 0
; loading file "/tmp/sbcl/src/code/late-globaldb.lisp":

; wrote /tmp/sbcl/obj/from-xc/src/code/late-globaldb.lisp-obj-scratch
; compilation finished in 0:00:00.003
fatal error encountered in SBCL pid 189515(tid 0x7f431257db80):
GC invariant lost, file "thread.c", line 1020

   0: HOST-SB-IMPL::%MAKE-HASH-TABLE, pc = 0x524713ed, fp = 0x7f43115cc408
   1: MAKE-HASH-TABLE, pc = 0x5211b419, fp = 0x7f43115cc4c8
   2: SB-C::DELETE-UNUSED-IR2-BLOCKS, pc = 0x530bf5f0, fp = 0x7f43115cc510
   3: SB-C::IR2-OPTIMIZE, pc = 0x530c3cdc, fp = 0x7f43115cc538
   4: SB-C::%COMPILE-COMPONENT, pc = 0x52da4318, fp = 0x7f43115cc5b8
   5: SB-C::COMPILE-COMPONENT, pc = 0x52da5411, fp = 0x7f43115cc5e8
   6: SB-C::SUB-COMPILE-TOPLEVEL-LAMBDAS, pc = 0x52daf586, fp = 0x7f43115cc618
   7: SB-C::COMPILE-TOPLEVEL-LAMBDAS, pc = 0x52daf780, fp = 0x7f43115cc668
   8: SB-C::COMPILE-TOPLEVEL, pc = 0x52dafc22, fp = 0x7f43115cc6b0
   9: SB-C::CONVERT-AND-MAYBE-COMPILE, pc = 0x52da9353, fp = 0x7f43115cc728
  10: SB-C::PROCESS-TOPLEVEL-FORM, pc = 0x52dadda1, fp = 0x7f43115cc7c0
  11: SB-C::PROCESS-TOPLEVEL-FORM, pc = 0x52dada38, fp = 0x7f43115cc858
  12: SB-C::PROCESS-TOPLEVEL-PROGN, pc = 0x52da9bf0, fp = 0x7f43115cc890
  13: SB-C::PROCESS-TOPLEVEL-FORM, pc = 0x52dad98c, fp = 0x7f43115cc928
  14: SB-C::PROCESS-TOPLEVEL-FORM, pc = 0x52dada38, fp = 0x7f43115cc9c0
  15: (LAMBDA (SB-KERNEL::FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-C::SUB-COMPILE-FILE), pc = 0x52db286c, fp = 0x7f43115ccae8
  16: SB-C::%DO-FORMS-FROM-INFO, pc = 0x52da72f2, fp = 0x7f43115ccbd0
  17: (LAMBDA () :IN SB-C::SUB-COMPILE-FILE), pc = 0x52db22f8, fp = 0x7f43115ccce8
  18: (FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT), pc = 0x52d9f3fc, fp = 0x7f43115ccdc0
  19: SB-C::SUB-COMPILE-FILE, pc = 0x52db1a58, fp = 0x7f43115ccf60
  20: SB-XC::COMPILE-FILE, pc = 0x52db3db3, fp = 0x7f43115cd080
  21: SB-COLD::COMPILE-STEM, pc = 0x52b783af, fp = 0x7f43115cd2b0
  22: (LAMBDA () :IN SB-COLD::TARGET-COMPILE-STEM), pc = 0x52b79d51, fp = 0x7f43115cd3e0
[89 frames in total]

line 1020 is "gc_assert(thread_state(p)==STATE_DEAD);"

I have a suspicion that --with-sb-show in the target features provokes this error, but even if parallelized build with sb-show is not supposed to work (I can't see why not), it's interesting that it causes this host failure as opposed to a build failure.

[Stas Boukarev (stassats)]

Does that happen reliably?

[Stas Boukarev (stassats)]

A test case. Stopping the finalizer thread doesn't mean the C part has exited and cleared all_threads by the time of fork()

(tagbody
 again
   (sb-thread:join-thread (sb-thread:make-thread (lambda () 0)))
   (let ((pid (sb-posix:posix-fork)))
     (cond ((zerop pid)
            (sb-ext:gc :full t)
            (sb-ext:exit))
           (t
            (sb-posix:wait)
            (go again)))))

[Douglas Katzman (dougk)]

This seems fixed.