use DBI fails and exits in some cases (embedded)

[Pierre Chifflier]

[please CC me, I'm not subscribed]

Hi,

I am developping an FTP server in C and use Perl as a
scripting/extension language. An interpreter is created and embedded
using perl_alloc() and friends. All seems to work (I am able to run
perl code correctly), except when trying to load some modules.

For ex, the code 'use DBI;' results in an error:
jui 19 12:14:39 <thread -1225401424> <- 'SITE test'
/home/pollux/DEL/sbin/wzdftpd: symbol lookup error:
/usr/lib/perl5/auto/DBI/DBI.so: undefined symbol: Perl_Tstack_sp_ptr

What is weird is that this errors calls exit() directy, it does use
the standard perl error mechanism (and so aborting the entire program,
not only the perl interpreter).

The C code causing the problem is simple:
val = eval_pv( "use DBI;", FALSE);

The exact same code works if placed in a separate (standalone)
program, so I suspect a side effect.

I have tried many things without success, and I explained my efforts
in details here:
http://www.wzdftpd.net/wiki/index.php/Scripting_in_Perl#symbol_lookup_error

I have put as many details as possible on this page, if something is
missing just tell me.

To resume, the problem seems related to the fact that the interpreter
is created and contained in a shared library (.so), loaded via
dlopen(). The exact same code, linked differently in the program,
works perfectly.

I am really hopping someone has run into this, any help would be much
appreciated.

Thanks,
Pierre

[Nick Ing-Simmons]

Post by Pierre Chifflier
[please CC me, I'm not subscribed]
jui 19 12:14:39 <thread -1225401424> <- 'SITE test'
/usr/lib/perl5/auto/DBI/DBI.so: undefined symbol: Perl_Tstack_sp_ptr
I have tried many things without success, and I explained my efforts
http://www.wzdftpd.net/wiki/index.php/Scripting_in_Perl#symbol_lookup_error
I have put as many details as possible on this page, if something is
missing just tell me.
To resume, the problem seems related to the fact that the interpreter
is created and contained in a shared library (.so), loaded via
dlopen(). The exact same code, linked differently in the program,
works perfectly.

Exactly. When a extension is loaded it looks for symbols in the
compiled in dependant libraries and in the "root" executable.
But the root is your main app not perl so doesn't export the perl symbols.

The "easy" fix is to dlopen() your perl.so with RTLD_GLOBAL in the flags
argument when you load it into your main application.
See dlopen() docs.

What (should) also work is explicitly linking perl extensions with
-L/-R/-l options which will find you perl.so
(OS runtime should then spot that your application already has that
loaded and use it - but this part hasn't always worked right.)

Post by Pierre Chifflier
what is weird is that this errors calls exit() directy, it does use
the standard perl error mechanism (and so aborting the entire program,
not only the perl interpreter).
val = eval_pv( "use DBI;", FALSE);
The exact same code works if placed in a separate (standalone)
program, so I suspect a side effect.
I am really hopping someone has run into this, any help would be much
appreciated.
Thanks,
Pierre

[Pierre Chifflier]

Post by Nick Ing-Simmons

Post by Pierre Chifflier
[please CC me, I'm not subscribed]
jui 19 12:14:39 <thread -1225401424> <- 'SITE test'
/usr/lib/perl5/auto/DBI/DBI.so: undefined symbol: Perl_Tstack_sp_ptr
I have tried many things without success, and I explained my efforts
http://www.wzdftpd.net/wiki/index.php/Scripting_in_Perl#symbol_lookup_error
I have put as many details as possible on this page, if something is
missing just tell me.
To resume, the problem seems related to the fact that the interpreter
is created and contained in a shared library (.so), loaded via
dlopen(). The exact same code, linked differently in the program,
works perfectly.

Exactly. When a extension is loaded it looks for symbols in the
compiled in dependant libraries and in the "root" executable.
But the root is your main app not perl so doesn't export the perl symbols.
The "easy" fix is to dlopen() your perl.so with RTLD_GLOBAL in the flags
argument when you load it into your main application.
See dlopen() docs.

Thanks a lot ! Using RTLD_GLOBAL solved my problem. Now all I have to
do is to check that it won't create problems, and that it works on all
supported platforms (mainly BSDs)

Post by Nick Ing-Simmons
What (should) also work is explicitly linking perl extensions with
-L/-R/-l options which will find you perl.so
(OS runtime should then spot that your application already has that
loaded and use it - but this part hasn't always worked right.)

Not sure I understand correctly this part. My link flags are:
`perl -MExtUtils::Embed -e ldopts`
which expands to:

-Wl,-E -L/usr/local/lib
/usr/lib/perl/5.8/auto/DynaLoader/DynaLoader.a
-L/usr/lib/perl/5.8/CORE -lperl -ldl -lm -lpthread -lc -lcrypt

Anyway, the RTLD_GLOBAL solution seems to work.

Thanks,
Pierre