I've been looking at the "libltdl and RTLD_GLOBAL" issue with embedding
perl in a dlopen'd plugin.
An instance of this with freeradius is #416266 (recently reassigned to
perl), and I see #327585 against openldap is another one.
To recap, the problem is that lt_dlopen() from the Debian system libltdl
has called dlopen(3) with RTLD_LOCAL instead of RTLD_GLOBAL ever since
#195821 was fixed. As the compiled XS modules aren't linked against
libperl, its symbols aren't exposed to them, resulting in errors like
'/usr/lib/perl/5.10/auto/Data/Dumper/Dumper.so: undefined symbol: Perl_sv_cmp'.
Observations:
- this problem isn't specific to perl and can easily be triggered with the
freeradius rlm_python module too [1]
- it's clearly possible to dlopen() compiled Perl modules from a dlopen'd
module if you don't use libltdl, see apache2+libapache2-mod-perl2 for
an example
- the XS modules are actually plugins in a private directory, not generic
shared libraries. Having unresolved symbols in a plugin without
a corresponding NEEDED entry seems to be very common, see for
example /usr/lib/apache2/modules, /usr/lib/python2.5/lib-dynload/,
/usr/lib/cdebconf etc.
- as noted in #327585, linking the XS shared objects against libperl
is potentially a problem on *i386, where /usr/bin/perl is statically
linked with libperl.a for performance reasons. (I don't have any data
about these performance reasons myself, I'm relying on hearsay and
/usr/share/doc/perl/README.Debian.gz here.)
While this does seem to work in a quick and limited test of mine,
it would bring in both libperl.a and libperl.so for all uses of
/usr/bin/perl that need XS modules, and I'm not sure which version of
the functions would get used later. If the PIC versions win, we'd be
giving away the performance benefit we got from static linking in the
first place.
At the very least, it would add 1.5M to the size of the perl-base
package on i386 AFAICS. I'm not sure how much the memory footprint of
the /usr/bin/perl invocations would increase.
Also note that we currently ship /usr/lib/libperl.a on all the
architectures, so everything that applies to the i386 /usr/bin/perl
case applies to anybody using the static library on the other archs too.
Given that i386 is still our most popular architecture, the other
proposed options don't seem very appealing either:
* only link the modules against libperl.so on the other architectures
(no fix for i386)
* link /usr/bin/perl dynamically on i386 too
(reduced performance in the very common case
for the benefit of a very uncommon case)
- it turns out libltdl nowadays does have an interface where you can
specify RTLD_GLOBAL. From the libtool Changelog.2007:
2007-05-08 Gary V. Vaughan <email address hidden>
Without this patch, lt_dlopen always opens modules with symbol
visibility set according to the underlying implementation.
Here, we add lt_dlopenadvise() to allow callers to request,
among other things, local or global symbol visibility from the
underlying dlloader:
Indeed, the attached proof of concept makes the freeradius problem go away
for me, and I expect openldap could work with something similar. (FWIW,
note that the trivial my_dlopenextglobal() function was adapted from the
libtool documentation, so it might be considered to be under the GFDL.)
Josip: based on the above, I think #416266 should be fixed in freeradius
and not in perl. If you agree, please reassign back yourself.
[1]: add python to the instantiate{} block in radiusd.conf
and something like
Hi,
I've been looking at the "libltdl and RTLD_GLOBAL" issue with embedding
perl in a dlopen'd plugin.
An instance of this with freeradius is #416266 (recently reassigned to
perl), and I see #327585 against openldap is another one.
To recap, the problem is that lt_dlopen() from the Debian system libltdl perl/5. 10/auto/ Data/Dumper/ Dumper. so: undefined symbol: Perl_sv_cmp'.
has called dlopen(3) with RTLD_LOCAL instead of RTLD_GLOBAL ever since
#195821 was fixed. As the compiled XS modules aren't linked against
libperl, its symbols aren't exposed to them, resulting in errors like
'/usr/lib/
Observations:
- this problem isn't specific to perl and can easily be triggered with the
freeradius rlm_python module too [1]
- it's clearly possible to dlopen() compiled Perl modules from a dlopen'd libapache2- mod-perl2 for
module if you don't use libltdl, see apache2+
an example
- the XS modules are actually plugins in a private directory, not generic apache2/ modules, /usr/lib/ python2. 5/lib-dynload/ ,
shared libraries. Having unresolved symbols in a plugin without
a corresponding NEEDED entry seems to be very common, see for
example /usr/lib/
/usr/lib/cdebconf etc.
- as noted in #327585, linking the XS shared objects against libperl share/doc/ perl/README. Debian. gz here.)
is potentially a problem on *i386, where /usr/bin/perl is statically
linked with libperl.a for performance reasons. (I don't have any data
about these performance reasons myself, I'm relying on hearsay and
/usr/
While this does seem to work in a quick and limited test of mine,
it would bring in both libperl.a and libperl.so for all uses of
/usr/bin/perl that need XS modules, and I'm not sure which version of
the functions would get used later. If the PIC versions win, we'd be
giving away the performance benefit we got from static linking in the
first place.
At the very least, it would add 1.5M to the size of the perl-base
package on i386 AFAICS. I'm not sure how much the memory footprint of
the /usr/bin/perl invocations would increase.
Also note that we currently ship /usr/lib/libperl.a on all the
architectures, so everything that applies to the i386 /usr/bin/perl
case applies to anybody using the static library on the other archs too.
Given that i386 is still our most popular architecture, the other
proposed options don't seem very appealing either:
* only link the modules against libperl.so on the other architectures
(no fix for i386)
* link /usr/bin/perl dynamically on i386 too
(reduced performance in the very common case
for the benefit of a very uncommon case)
- it turns out libltdl nowadays does have an interface where you can
specify RTLD_GLOBAL. From the libtool Changelog.2007:
2007-05-08 Gary V. Vaughan <email address hidden>
Without this patch, lt_dlopen always opens modules with symbol
visibility set according to the underlying implementation.
Here, we add lt_dlopenadvise() to allow callers to request,
among other things, local or global symbol visibility from the
underlying dlloader:
Indeed, the attached proof of concept makes the freeradius problem go away bal() function was adapted from the
for me, and I expect openldap could work with something similar. (FWIW,
note that the trivial my_dlopenextglo
libtool documentation, so it might be considered to be under the GFDL.)
Josip: based on the above, I think #416266 should be fixed in freeradius
and not in perl. If you agree, please reassign back yourself.
[1]: add python to the instantiate{} block in radiusd.conf
and something like
# cat /etc/freeradius /modules/ python
mod_instantia te = radiusd_test
func_ instantiate = instantiate lib/python2. 5/site- packages/ radiusd_ test.py
python {
}
# cat /usr/local/
import sys
import socket
def instantiate(test): stderr. write(" hello, world!")
sys.
and you get
rlm_python: EXCEPT: <type 'exceptions. ImportError' >: /usr/lib/ python2. 5/lib-dynload/ _socket. so: undefined symbol: PyExc_ValueError python_ load_function: failed to import python function 'radiusd_ test.instantiat e'
rlm_python:
--
Niko Tyni <email address hidden>