darwin dlshim needs to die

[garaemon]

on snow leopard(darwin, x86-64), i found sbcl cannot load some shared objects.

test-case is:
(load-shared-object "/usr/X11R6/lib/libX11.dylib") ;; it works on 1.36.13 sbcl
(load-shared-object "/usr/X11R6/lib/libGL.dylib") ;; it does not work

sbcl does not use dlopen et.al. of <dlfcn.h> but use <mach-o/dyld.h> and
emurating dlopen et.al. itself at src/runtime/darwin-dlshim.c .

I think we should use dlopen et. al. of <dlfcn.> provided by darwin gcc, so i write some patches
to use them.

my configurations are:
$ uname -a
Darwin xxxx 10.2.0 Darwin Kernel Version 10.2.0: Tue Nov 3 10:37:10 PST 2009; root:xnu-1486.2.11~1/RELEASE_I386 i386
$ sbcl --version
=> SBCL 1.0.36.13
$ sbcl
* (machine-type)
"X86-64"
* *features*
(:SB-THREAD :ANSI-CL :COMMON-LISP :SBCL :SB-DOC :SB-TEST :SB-LDB
 :SB-PACKAGE-LOCKS :SB-UNICODE :SB-EVAL :SB-SOURCE-LOCATIONS
 :IEEE-FLOATING-POINT :X86-64 :INODE64 :UNIX :MACH-O :BSD :DARWIN
 :MACH-EXCEPTION-HANDLER :SB-LUTEX :UD2-BREAKPOINTS :GENCGC
 :STACK-GROWS-DOWNWARD-NOT-UPWARD :C-STACK-IS-CONTROL-STACK :LINKAGE-TABLE
 :COMPARE-AND-SWAP-VOPS :UNWIND-TO-FRAME-AND-CALL-VOP :RAW-INSTANCE-INIT-VOPS
 :STACK-ALLOCATABLE-CLOSURES :STACK-ALLOCATABLE-VECTORS
 :STACK-ALLOCATABLE-LISTS :STACK-ALLOCATABLE-FIXED-OBJECTS :ALIEN-CALLBACKS
 :CYCLE-COUNTER :COMPLEX-FLOAT-VOPS :FLOAT-EQL-VOPS :INLINE-CONSTANTS
 :OS-PROVIDES-DLOPEN :OS-PROVIDES-DLADDR :OS-PROVIDES-PUTWC
 :OS-PROVIDES-BLKSIZE-T :OS-PROVIDES-SUSECONDS-T)

patches are (ugly ones):
Index: sb-bsd-sockets/defpackage.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/contrib/sb-bsd-sockets/defpackage.lisp,v
retrieving revision 1.14
diff -r1.14 defpackage.lisp
76a77,79
>
> #+darwin
> (load-shared-object "/usr/lib/libc.dylib") ;;why?
Index: contrib/sb-posix/defpackage.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/contrib/sb-posix/defpackage.lisp,v
retrieving revision 1.16
diff -r1.16 defpackage.lisp
26a27,29
>
> #+darwin
> (sb-alien:load-shared-object "/usr/lib/libc.dylib") ;;why?
Index: src/code/unix-foreign-load.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/src/code/unix-foreign-load.lisp,v
retrieving revision 1.1
diff -r1.1 unix-foreign-load.lisp
34a35
>
69c70,71
< (let* ((extern (extern-alien-name symbol))
---
> (let* ((extern #!-mach-o (extern-alien-name symbol)
> #!+mach-o (coerce symbol 'base-string)) ;; extern-alien-name provides "_foo" on darwin
Index: src/runtime/Config.x86-64-darwin9+
===================================================================
RCS file: /cvsroot/sbcl/sbcl/src/runtime/Config.x86-64-darwin9+,v
retrieving revision 1.1
diff -r1.1 Config.x86-64-darwin9+
14c14,15
< OS_SRC = bsd-os.c x86-64-bsd-os.c darwin-os.c x86-64-darwin-os.c darwin-dlshim.c darwin-langinfo.c
---
> #OS_SRC = bsd-os.c x86-64-bsd-os.c darwin-os.c x86-64-darwin-os.c darwin-dlshim.c darwin-langinfo.c
> OS_SRC = bsd-os.c x86-64-bsd-os.c darwin-os.c x86-64-darwin-os.c darwin-langinfo.c
Index: src/runtime/x86-64-darwin-os.c
===================================================================
RCS file: /cvsroot/sbcl/sbcl/src/runtime/x86-64-darwin-os.c,v
retrieving revision 1.14
diff -r1.14 x86-64-darwin-os.c
25a26,29
> // added by garaemon, required?
> #include <dlfcn.h>
> #include <sys/wait.h>
> #include <mach-o/dyld.h>
688a693,697
> //dummy function
> /* void darwin_waitpid(int pid, int* status, int options) { */
> /* waitpid(pid, status, options); */
> /* } */
>
Index: tools-for-build/ldso-stubs.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/tools-for-build/ldso-stubs.lisp,v
retrieving revision 1.24
diff -r1.24 ldso-stubs.lisp
327,331c327,337
< #!-darwin
< '("dlclose"
< "dlerror"
< "dlopen"
< "dlsym")
---
> ;; #!-darwin
> '("dlclose"
> "dlerror"
> "dlopen"
> "dlsym")
> #!+darwin
> '("waitpid"
> "ptsname"
> "grantpt"
> "unlockpt")
> ;; for bsd-sockets...?

-- garaemon

[Nikodemus Siivola (nikodemus)]

Thanks for the report!

Not a Snow Leopard issue. On plain old leopard:

* (load-shared-object "/usr/X11R6/lib/libGL.dylib")
CORRUPTION WARNING in SBCL pid 11721:
Memory fault at 13 (pc=0x7fff5fc0d193, sp=0x31ff550)

...I might have confirmed this much earlier if the subject had not referred to 10.6, which I don't have access to. Moral: don't speculate. :)

[Nikodemus Siivola (nikodemus)]

(The dlshim was implemented at a time when OS X didn't provide a fully functional dlopen &co.)

[Cyrus Harmon (ch-launchpad)]

Fixed in 1.0.39.3.

The dlshim is now an optional feature (build with :dlshim). This can probably go away entirely, but some old versions of darwin (that we probably don't support anymore) may require the dlshim.