Ubuntu

JNI unable to find libpam.so

Reported by James Page on 2011-03-18
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libjna-java (Debian)
Fix Released
Unknown
libjna-java (Ubuntu)
High
James Page
openjdk-6 (Ubuntu)
High
Matthias Klose

Bug Description

I'm currently packaging Jenkins for Ubuntu; this application has native authentication integration through the use of PAM. This was working fine i.e. the java native interface dispatcher was able to find libpam.so in /lib and all was happy.

I've just updated my test system and native authentication no longer works; the JNI dispatcher can't find the library. I have done a bit of digging and it looks like libpam.so.* is installed to /lib/i386-linux-gnu instead of /lib (as on Maverick and possibly earlier Natty releases?). This directory is not on the java.library.path by default which I suspect may be the issue (see below for default java library path):

/usr/lib/jvm/java-6-openjdk/jre/lib/i386/client:/usr/lib/jvm/java-6-openjdk/jre/lib/i386:/usr/lib/jvm/java-6-openjdk/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/jni:/lib:/usr/lib

So I'm not sure whether this is a bug in pam or openjdk but I guess it could be either.

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: libpam0g 1.1.2-2ubuntu4
ProcVersionSignature: Ubuntu 2.6.38-7.35-generic-pae 2.6.38
Uname: Linux 2.6.38-7-generic-pae i686
Architecture: i386
Date: Fri Mar 18 14:54:45 2011
InstallationMedia: Ubuntu-Server 11.04 "Natty Narwhal" - Alpha i386 (20110301.4)
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: pam
UpgradeStatus: No upgrade log present (probably fresh install)

James Page (james-page) wrote :
James Page (james-page) wrote :

I'm guessing that this is related to bug 733501.

Steve Langasek (vorlon) wrote :

Hi James,

> I'm guessing that this is related to bug 733501.

Yes, this is related to 733501. libpam has moved paths to the multiarch-compliant directory, and openjdk needs to be updated to know about these directories.

In general it would be nice if openjdk weren't hard-coding its own path... if it were simply falling back to the default search path from libdl, this transition should have been transparent to openjdk.

affects: pam (Ubuntu) → openjdk-6 (Ubuntu)
Changed in openjdk-6 (Ubuntu):
importance: Undecided → High
status: New → Triaged
tags: added: multiarch
Steve Langasek (vorlon) on 2011-03-18
Changed in openjdk-6 (Ubuntu):
milestone: none → ubuntu-11.04-beta-1
Steve Langasek (vorlon) wrote :

Can you give me a test case that lets me reproduce the failure? I've found where "DEFAULT_LIBPATH" is defined in openjdk/hotspot/src/os/linux/vm/os_linux.cpp, but as I said I would really like to fix this to use the system library path instead of its own copy. but there are an awful lot of calls to dlopen() in this code and I'm not sure which one is the source of this problem. (I currently suspect jdk/src/share/classes/java/lang/ClassLoader.java, but I could be wrong.)

Steve Langasek (vorlon) wrote :

Attached is a tentative patch for this issue (completely untested). Removing "/lib:/usr/lib" from the DEFAULT_LIBPATH is not strictly necessary, although leaving them here means they'll be searched twice (once explicitly, once as part of ld.so's default search path) and we don't really want them to appear in front of the multiarch dirs on the search path. But it also forces the issue; if there's other code relying on java.lib.path to be complete, that code will start failing immediately even if the libraries they're looking for haven't been converted for multiarch. So this needs to be weighed carefully given where we are in the release cycle.

(BTW, Matthias, the openjdk-6 source package has a Vcs-Bzr field pointing at a branch that's not up to date. Did you forget to push?)

Changed in openjdk-6 (Ubuntu):
assignee: nobody → Matthias Klose (doko)
Steve Langasek (vorlon) wrote :

Oops, debdiff doesn't work so well, this needs to be applied after the default-libpath patch. Updated patch attached.

Steve Langasek (vorlon) wrote :

And there was a missing space in the patch, so it didn't build either. Here's an updated patch; I guess I'll have one more update once it manages to build and I figure out what doesn't work about it at runtime. ;)

tags: added: patch
Steve Langasek (vorlon) wrote :

Attaching the diff between the previous jtreg-summary log in the package, and the one resulting from a local test build on my system with the patch applied. I'm not sure how to interpret this - the number of passed tests has increased, but there are now two errors which weren't there before. Matthias, are these errors among the expected "phase of the moon" variations?

James Page (james-page) wrote :

Steve

So I tried out the patch (took some time to build ppa:james-page/junk) and did some more digging when it did not appear to work.

I managed to capture a stacktrace when the issue occured:

java.lang.UnsatisfiedLinkError: Unable to load library 'pam': libpam.so: cannot open shared object file: No such file or directory
 at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:164)
 at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:237)
 at com.sun.jna.Library$Handler.(Library.java:140)
 at com.sun.jna.Native.loadLibrary(Native.java:374)
 at com.sun.jna.Native.loadLibrary(Native.java:359)
 at org.jvnet.libpam.impl.PAMLibrary.(PAMLibrary.java:132)
 at sun.misc.Unsafe.ensureClassInitialized(Native Method)
 at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
 at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:140)
 at java.lang.reflect.Field.acquireFieldAccessor(Field.java:936)
 at java.lang.reflect.Field.getFieldAccessor(Field.java:917)
 at java.lang.reflect.Field.get(Field.java:376)
 at com.sun.jna.Native.loadLibraryInstance(Native.java:398)
 at com.sun.jna.Native.getStructureAlignment(Native.java:507)
 at com.sun.jna.Structure.setAlignType(Structure.java:191)
 at com.sun.jna.Structure.(Structure.java:149)
 at com.sun.jna.Structure.(Structure.java:145)
 at com.sun.jna.Structure.(Structure.java:141)
 at com.sun.jna.Structure.(Structure.java:132)

So it looks like the issue (or maybe one of them) is actually in libjna-java; I'm pretty sure that openjdk may exhibit the same symptoms but the issue reading libpam.so is generated outside of openjdk.

I'll look into the specific library to see what its doing - suspect it something horrible with guessed paths but it may be related to the java.library.path variable.

James Page (james-page) wrote :

This is odd in itself as although libjna-java does munge a library path together, if the library is not found in the search path it should just let dlopen do it thing.

This does not appear to be working.

I'm going to put a fresh pair of eyes on this tomorrow morning....

James Page (james-page) on 2011-03-22
Changed in libjna-java (Ubuntu):
status: New → Triaged
importance: Undecided → High
assignee: nobody → James Page (james-page)
James Page (james-page) wrote :

libjna-java has to deal with a number of oddities including unversioned library names etc.. so I will write a patch to correctly form the search path for multiarch enabled systems.

Changed in libjna-java (Ubuntu):
milestone: none → ubuntu-11.04-beta-1
James Page (james-page) wrote :

Matthias

I'm not sure of the impact of this issue on openjdk-6; its definitely possible that applications that rely on JNI integration might not work after the multi-arch change as some of the unversioned (libpam.so for example) native libraries have moved out into -dev packages.

The issue I encountered was due to a) paths changing and libjna-java not dealing with this and b) the unversioned libraries in /lib32 disappearing with the multiarch change.

Cheers

James

Steve Langasek (vorlon) wrote :

Looking at the code in libjna-java, it appears that it already supports fallback to the system path. If the caller is fixed to (properly) look for libpam.so.0 instead of libpam.so, does that solve the issue?

> its definitely possible that applications that rely on JNI integration might
> not work after the multi-arch change as some of the unversioned
> (libpam.so for example) native libraries have moved out into -dev packages.

This has nothing to do with multiarch. the unversioned .so was *never* in the runtime library package, and is not the filename that anything should be referencing at runtime.

James Page (james-page) wrote :

Steve

OK - my error on libraries moving out into -dev - they where actually in /lib32 (see output from maverick system below)

jamespage@hendrix:~/src/natty/jna/libjna-java-multiarch-fix/src/com/sun/jna$ ldconfig -p | grep libpam.so
 libpam.so.0 (libc6,x86-64) => /lib/libpam.so.0
 libpam.so.0 (libc6) => /lib32/libpam.so.0
 libpam.so (libc6) => /lib32/libpam.so
jamespage@hendrix:~/src/natty/jna/libjna-java-multiarch-fix/src/com/sun/jna$

So dlopen('libpam.so') did work before - but only because libpam.so was in ld.so.cache courtesy of ia32-libs.

I've made a few further comments in the linked merge proposal with regards to JNA behaviour

James Page (james-page) wrote :

Small Java utility to test JDK native library loading:

javac NativeTest.java
java NativeTest <libname>

e.g.

java NativeTest pam

Outputs java.library.path and sun.boot.library.path and the mapping that the jvm has done interms of libname to actual native library name.

James Page (james-page) wrote :

Steve

I also spent some time looking at the System.loadLibrary method provided by the JVM to load libraries (see attached NativeTest.java).

Basically this call will only accept an unqualified library name - 'pam' for example - which it then maps into a versionless 'libpam.so' library name - which does not exist on Ubuntu Natty! I can see anyway to inject a version number alongside the library name.

As I put in my previous comment, I don't know exactly how much of an issue this is i.e. I can't easily tell how many packages rely on version-less library links being present.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package openjdk-6 - 6b22-1.10.1~pre1-0ubuntu1

---------------
openjdk-6 (6b22-1.10.1~pre1-0ubuntu1) natty; urgency=low

  * Update from the IcedTea6-1.10 release branch (20110325).
  * Add multiarch directories to the default library path. LP: #737603.
 -- Matthias Klose <email address hidden> Fri, 25 Mar 2011 16:33:57 +0100

Changed in openjdk-6 (Ubuntu):
status: Triaged → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libjna-java - 3.2.4-2ubuntu1

---------------
libjna-java (3.2.4-2ubuntu1) natty; urgency=low

  * Fix native library search to be multi-arch compatible (LP: #737603):
    - debian/patches/07-multiarch-support.patch : figure out
      platform and form search path accordingly.
 -- James Page <email address hidden> Tue, 22 Mar 2011 08:57:17 +0000

Changed in libjna-java (Ubuntu):
status: Triaged → Fix Released
Matthias Klose (doko) wrote :

mulitarch paths on powerpc and ppc64 are wrong:

$ dpkg-architecture -appc64 -qDEB_HOST_MULTIARCH
powerpc64-linux-gnu
$ dpkg-architecture -apowerpc -qDEB_HOST_MULTIARCH
powerpc-linux-gnu

Changed in libjna-java (Ubuntu):
status: Fix Released → Confirmed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libjna-java - 3.2.4-2ubuntu2

---------------
libjna-java (3.2.4-2ubuntu2) natty; urgency=low

  * Fix incorrect generation of PowerPC multi-arch paths (LP: #737603).
 -- James Page <email address hidden> Mon, 28 Mar 2011 09:27:48 +0100

Changed in libjna-java (Ubuntu):
status: Confirmed → Fix Released
James Page (james-page) wrote :

JNA fixed submitted back to upstream project - http://java.net/jira/browse/JNA-184

Changed in libjna-java (Debian):
status: Unknown → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.