LispWorks on OS X cannot find libs in standard paths

[Robert Smith]

During the development of CL-CHARMS (https://github.com/HiTECNOLOGYs/cl-charms), I was unable to get LispWorks 6.1.1 on OS X 10.10 to find the OS-provided library /usr/lib/ncurses.dylib. The code before was:

    (cffi:define-foreign-library libcurses
      (:darwin (:or "libcurses.dylib" "libncurses.dylib"))
      (:unix (:or "libncurses.so.5" "libcurses"))
      (t (:default "libcurses")))

    (cffi:use-foreign-library libcurses)

SBCL loads this just fine without having to modify any paths or variables.

In an effort to workaround, I tried

    (push "/usr/lib" cffi:*foreign-library-directories*)

This also didn't work.

The workaround I found to work currently is to specify the full path in the DEFINE-FOREIGN-LIBRARY macro.

This is for CFFI 0.14.0, provided by Quicklisp.

[Luís Oliveira (luismbo)]

Signed binaries on OS X have that effect on dlopen(). Try pushing /usr/lib/
with a trailing slash. That should work. Maybe CFFI should provide this
workaround out of the box?

On Sat, Jan 24, 2015, 10:05 Robert Smith <email address hidden> wrote:

> Public bug reported:
>
> During the development of CL-CHARMS (https://github.com/HiTECNOLOGYs/cl-
> charms), I was unable to get LispWorks 6.1.1 on OS X 10.10 to find the
> OS-provided library /usr/lib/ncurses.dylib. The code before was:
>
> (cffi:define-foreign-library libcurses
> (:darwin (:or "libcurses.dylib" "libncurses.dylib"))
> (:unix (:or "libncurses.so.5" "libcurses"))
> (t (:default "libcurses")))
>
> (cffi:use-foreign-library libcurses)
>
> SBCL loads this just fine without having to modify any paths or
> variables.
>
> In an effort to workaround, I tried
>
> (push "/usr/lib" cffi:*foreign-library-directories*)
>
> This also didn't work.
>
> The workaround I found to work currently is to specify the full path in
> the DEFINE-FOREIGN-LIBRARY macro.
>
> This is for CFFI 0.14.0, provided by Quicklisp.
>
> ** Affects: cffi
> Importance: Undecided
> Status: New
>
>
> ** Tags: lispworks
>
> ** Tags added: lispworks
>
> --
> You received this bug notification because you are subscribed to CFFI.
> https://bugs.launchpad.net/bugs/1414277
>
> Title:
> LispWorks on OS X cannot find libs in standard paths
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cffi/+bug/1414277/+subscriptions
>

[Robert Smith (quad)]

Adding a trailing slash did make it work. I suspect this has to do with the underlying method for resolving the location of libraries for LispWorks.

From CFFI's perspective, it would be nice if /usr/lib/ was known, since it seems to be known by SBCL.

[Luís Oliveira (luismbo)]

The trailing slash bit has to do with how CL's pathnames work. #p"/usr/lib" denotes a file named "lib" in the "/usr" directory, whereas #p"/usr/lib/" denotes the "/usr/lib" directory.

    CL-USER> (pathname-directory "/usr/lib")
    (:ABSOLUTE "usr")

    CL-USER> (pathname-directory "/usr/lib/")
    (:ABSOLUTE "usr" "lib")

That's what was tripping CFFI's lookup mechanism.

The difference you're seeing between Lispworks and SBCL exists because Lispworks's binary is codesigned and SBCL's isn't.

See the second note within the SEARCHING section in <https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dlopen.3.html>.

[Luís Oliveira (luismbo)]

The attached patch reimplements OS X's search strategy in non-codesigned binaries. Please let me know if it works for you.

[Robert Smith (quad)]

That patch worked for me, with one modification: You need to add #:UIOP as a :DEPENDS-ON in cffi.asd. Otherwise, perfect.

[Luís Oliveira (luismbo)]

Refactored the patch a little bit and committed. I hope I didn't break anything because I don't have a good way to test it. (It'd would have been nice to refactor the search strategy bits so they can be unit tested... Maybe next time!)

I've also added a note to the manual pointing out the nuances of CL's pathnames. (I considered using UIOP:ENSURE-DIRECTORY-PATHNAME instead, but the distinction is important in general, so I'm always hesitant to insert potentially confusing DWIM-style behaviour. Feel free to try and convince me otherwise!)