XS module wrapping a C++ library - compilation pb

[Christophe Massaloux]

Hi to all XS experts,
I face a strange pb while compiling my XS module, and really need some
support/help/advice to solve it.

I'm running Ubuntu 8.04 Hardy, kernel 2.6.24-19-generic
My Perl is v5.8.8 built for i486-linux-gnu-thread-multi
My compiler is g++ (GCC) 4.2.3 (Ubuntu 4.2.3-2ubuntu7)
My C+ STL is libstdc++6-4.2-dev version 4.2.3-2ubuntu7

I have written a C++ library to handle mutiplexing/demultiplexing of
digital video streams. Its name is libmosmux.
The library works fine as long as I stay in the C++ world.
I can compile it, and can link it to C++ executable. The resulting
executables do run without any pb.

Now I want to wrap that library in a Perl XS module named MosMux.pm
I have written a Makefile.PL, the MosMux.pm module and the MosMux.xs
file as explained in the tutorials I have read.
there are also a ppport.h and a typemap files.

When I finally launch "make", I got the following messages:

$ make
cp lib/MOS/MosMux.pm blib/lib/MOS/MosMux.pm
AutoSplitting blib/lib/MOS/MosMux.pm (blib/lib/auto/MosMux)
/usr/bin/perl /usr/share/perl5/ExtUtils/xsubpp -C++
-typemap /usr/share/perl/5.8/ExtUtils/typemap -typemap typemap -typemap
typemap MosMux.xs > MosMux.xsc && mv MosMux.xsc MosMux.c
Please specify prototyping behavior for MosMux.xs (see perlxs manual)
g++ -c -I. -I../../libmosmux -I../../DTAPI/Include -D_REENTRANT
-D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2
-DVERSION=\"0.01\" -DXS_VERSION=\"0.01\" -fPIC
"-I/usr/lib/perl/5.8/CORE" MosMux.c
In file included from MosMux.xs:14:
./../../libmosmux/libmosmux.h:136:1: warning: "VERSION" redefined
<command-line>: warning: this is the location of the previous definition
In file included from /usr/include/c++/4.2/bits/basic_ios.h:44,
from /usr/include/c++/4.2/ios:50,
from /usr/include/c++/4.2/istream:44,
from /usr/include/c++/4.2/fstream:45,
from ./../../libmosmux/libmosmux.h:228,
from MosMux.xs:14:
/usr/include/c++/4.2/bits/locale_facets.h:4420:40: error: macro
"do_open" requires 7 arguments, but only 2 given
/usr/include/c++/4.2/bits/locale_facets.h:4467:34: error: macro
"do_close" requires 2 arguments, but only 1 given
/usr/include/c++/4.2/bits/locale_facets.h:4486:55: error: macro
"do_open" requires 7 arguments, but only 2 given
/usr/include/c++/4.2/bits/locale_facets.h:4513:23: error: macro
"do_close" requires 2 arguments, but only 1 given
In file included from /usr/include/c++/4.2/bits/locale_facets.h:4599,
from /usr/include/c++/4.2/bits/basic_ios.h:44,
from /usr/include/c++/4.2/ios:50,
from /usr/include/c++/4.2/istream:44,
from /usr/include/c++/4.2/fstream:45,
from ./../../libmosmux/libmosmux.h:228,
from MosMux.xs:14:
/usr/include/c++/4.2/i486-linux-gnu/bits/messages_members.h:70:38:
error: macro "do_open" requires 7 arguments, but only 2 given
/usr/include/c++/4.2/i486-linux-gnu/bits/messages_members.h:85:23:
error: macro "do_open" requires 7 arguments, but only 2 given
/usr/include/c++/4.2/i486-linux-gnu/bits/messages_members.h:95:39:
error: macro "do_close" requires 2 arguments, but only 1 given
In file included from /usr/include/c++/4.2/bits/basic_ios.h:44,
from /usr/include/c++/4.2/ios:50,
from /usr/include/c++/4.2/istream:44,
from /usr/include/c++/4.2/fstream:45,
from ./../../libmosmux/libmosmux.h:228,
from MosMux.xs:14:
/usr/include/c++/4.2/bits/locale_facets.h:4486: error: ‘do_open’
declared as a ‘virtual’ field
/usr/include/c++/4.2/bits/locale_facets.h:4486: error: expected ‘;’
before ‘const’
/usr/include/c++/4.2/bits/locale_facets.h:4513: error: variable or field
‘do_close’ declared void
/usr/include/c++/4.2/bits/locale_facets.h:4513: error: expected ‘;’
before ‘const’
In file included from /usr/include/c++/4.2/bits/locale_facets.h:4599,
from /usr/include/c++/4.2/bits/basic_ios.h:44,
from /usr/include/c++/4.2/ios:50,
from /usr/include/c++/4.2/istream:44,
from /usr/include/c++/4.2/fstream:45,
from ./../../libmosmux/libmosmux.h:228,
from MosMux.xs:14:
/usr/include/c++/4.2/i486-linux-gnu/bits/messages_members.h:84: error:
expected initializer before ‘const’
/usr/include/c++/4.2/i486-linux-gnu/bits/messages_members.h:95: error:
expected initializer before ‘const’
MosMux.c: In function ‘void boot_MOS__MosMux(PerlInterpreter*, CV*)’:
MosMux.c:257: warning: deprecated conversion from string constant to
‘char*’
MosMux.c:263: warning: deprecated conversion from string constant to
‘char*’
MosMux.c:264: warning: deprecated conversion from string constant to
‘char*’
MosMux.c:265: warning: deprecated conversion from string constant to
‘char*’
MosMux.c:266: warning: deprecated conversion from string constant to
‘char*’
make: *** [MosMux.o] Error 1



Forget about the warnings for deprecated conversion. I have to fix it,
but it's not the point.

The real pb are the various errors occuring from included files from the
STL lib.
I don't understand why I got these errors when compiling the XS module
while I don't have any when compiling the C++ libmosmux library.

I'd really appreciate any hint that could lead me to the light..

Thanks in advance

Christophe
***@canal-overseas.com




Here are the various files mentioned:

Makefile.PL
============
use 5.008008;
use ExtUtils::MakeMaker;
# See lib/ExtUtils/MakeMaker.pm for details of how to influence
# the contents of the Makefile that is written.
$CC = 'g++';

WriteMakefile(
NAME => 'MOS::MosMux',
VERSION_FROM => 'lib/MOS/MosMux.pm',
PREREQ_PM => {
'Test::More' => 0,
},
($] >= 5.005 ? ## Add these new keywords supported since 5.005
(ABSTRACT_FROM => 'lib/MOS/MosMux.pm', # retrieve abstract from
module
AUTHOR => 'Christophe Massaloux
<***@canal-overseas.com>') : ()),
LIBS => [''], # e.g., '-lm'
DEFINE => '', # e.g., '-DHAVE_SOMETHING'
INC => '-I. -I../../libmosmux -I../../DTAPI/Include',
# e.g., '-I. -I/usr/include/other'
'CC' => $CC,
'LD' => '$(CC)',
XSOPT => '-C++',
TYPEMAPS => ['typemap'],
MYEXTLIB => '../../libmosmux/libmosmux$(LIB_EXT)',
# Un-comment this if you add C files to link with later:
OBJECT => '$(O_FILES)', # link all the C files too
dist => { COMPRESS => 'gzip -9f', SUFFIX => 'gz', },
clean => { FILES => 'MOS-MosMux-*' },
);

if (eval {require ExtUtils::Constant; 1}) {
# If you edit these definitions to change the constants used by this
module,
# you will need to use the generated const-c.inc and const-xs.inc
# files to replace their "fallback" counterparts before distributing
your
# changes.
my @names = (qw());
ExtUtils::Constant::WriteConstants(
NAME => 'MosMux',
NAMES => \@names,
DEFAULT_TYPE => 'IV',
C_FILE => 'const-c.inc',
XS_FILE => 'const-xs.inc',
);

}
else {
use File::Copy;
use File::Spec;
foreach my $file ('const-c.inc', 'const-xs.inc') {
my $fallback = File::Spec->catfile('fallback', $file);
copy ($fallback, $file) or die "Can't copy $fallback to $file: $!";
}
}

sub MY::postamble {
'
$(MYEXTLIB): ../../libmosmux/Makefile
cd ../../libmosmux && $(MAKE) $(PASSTHRU)
';
}


MosMux.pm:
==========
package MosMux;
use 5.008008;
use strict;
use warnings;
use Carp;
require Exporter;
use AutoLoader;

our @ISA = qw(Exporter);
our %EXPORT_TAGS = ( 'all' => [ qw() ] );
our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
our @EXPORT = qw();
our $VERSION = '0.01';
sub AUTOLOAD {
my $constname;
our $AUTOLOAD;
($constname = $AUTOLOAD) =~ s/.*:://;
croak "&MosMux::constant not defined" if $constname eq 'constant';
my ($error, $val) = constant($constname);
if ($error) { croak $error; }
{
no strict 'refs';
# Fixed between 5.005_53 and 5.005_61
#XXX if ($] >= 5.00561) {
#XXX *$AUTOLOAD = sub () { $val };
#XXX }
#XXX else {
*$AUTOLOAD = sub { $val };
#XXX }
}
goto &$AUTOLOAD;
}

require XSLoader;
XSLoader::load('MosMux', $VERSION);

1;



MosMux.xs:
Note: the "sectionRead" class is just a wrapper for the needed features
of the lib
==========
#ifdef __cplusplus
extern "C" {
#endif
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "ppport.h"
#include "const-c.inc"
#ifdef __cplusplus
}
#endif

#undef list
#include <../../libmosmux/libmosmux.h>

using namespace std;
using namespace mosmux;

class sectionRead
{
private:
MosMuxMultiplexer* m_mux;
MosMuxSaver* m_saver;
MosMuxOutput* m_out;

public:
sectionRead(int BitRateIn, char* inputTSFileName, int pid, int
tid, bool hasLSN, bool hasCRC32) {
MosMuxInput* in1 = new MosMuxTSFileInput( BitRateIn,
inputTSFileName );
m_out = new MosMuxMonoSectionStringOutput(BitRateIn, pid,
tid, hasLSN, hasCRC32, 2);
m_saver =
((MosMuxSectionExporter*)(m_out->getExporter()))->getSaver();
m_mux = new MosMuxMultiplexer();
m_mux->addInput(in1);
m_mux->addOutput(m_out);
m_mux->start();
}
~sectionRead(){}

std::string read() {
m_mux->run();
m_out->flush();
std::string section =
((MosMuxSaver2String*)(m_saver))->getSectionsString();
((MosMuxSaver2String*)(m_saver))->clearSectionsString();
return section;
}
};


MODULE = MOS::MosMux PACKAGE = MOS::MosMux

INCLUDE: const-xs.inc

sectionRead *
sectionRead::new(BitRateIn, inputTSFileName, pid, tid, hasLSN, hasCRC32)
int BitRateIn
char* inputTSFileName
int pid
int tid
bool hasLSN
bool hasCRC32

void
sectionRead::DESTROY()

SV *
sectionRead::read()
INIT:
std::string retstring;
CODE:
retstring = THIS->read();
RETVAL = newSVpvn(retstring.c_str(), retstring.size());
OUTPUT:
RETVAL



typemap:
========
#ifdef __cplusplus
extern "C" {
#endif
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"
#include "ppport.h"
#include "const-c.inc"
#ifdef __cplusplus
}
#endif

#undef list
#include <../../libmosmux/libmosmux.h>

using namespace std;
using namespace mosmux;

class sectionRead
{
private:
MosMuxMultiplexer* m_mux;
MosMuxSaver* m_saver;
MosMuxOutput* m_out;

public:
sectionRead(int BitRateIn, char* inputTSFileName, int pid, int
tid, bool hasLSN, bool hasCRC32) {
MosMuxInput* in1 = new MosMuxTSFileInput( BitRateIn,
inputTSFileName );
m_out = new MosMuxMonoSectionStringOutput(BitRateIn, pid,
tid, hasLSN, hasCRC32, 2);
m_saver =
((MosMuxSectionExporter*)(m_out->getExporter()))->getSaver();
m_mux = new MosMuxMultiplexer();
m_mux->addInput(in1);
m_mux->addOutput(m_out);
m_mux->start();
}
~sectionRead(){}

std::string read() {
m_mux->run();
m_out->flush();
std::string section =
((MosMuxSaver2String*)(m_saver))->getSectionsString();
((MosMuxSaver2String*)(m_saver))->clearSectionsString();
return section;
}
};


MODULE = MOS::MosMux PACKAGE = MOS::MosMux

INCLUDE: const-xs.inc

sectionRead *
sectionRead::new(BitRateIn, inputTSFileName, pid, tid, hasLSN, hasCRC32)
int BitRateIn
char* inputTSFileName
int pid
int tid
bool hasLSN
bool hasCRC32

void
sectionRead::DESTROY()

SV *
sectionRead::read()
INIT:
std::string retstring;
CODE:
retstring = THIS->read();
RETVAL = newSVpvn(retstring.c_str(), retstring.size());
OUTPUT:
RETVAL

[Sisyphus]

----- Original Message -----
From: "Christophe Massaloux" <***@canal-overseas.com>
.
.

Post by Christophe Massaloux
I don't understand why I got these errors when compiling the XS module
while I don't have any when compiling the C++ libmosmux library.

Looks like this is the result of a clash with the perl symbols do_close
(Perl_do_close) and do_open (Perl_do_open) which, upon inspection of the
perl source's doio.c, *do* take 2 and 7 arguments respectively.

I don't have any advice on how to work around this particular problem.

Cheers,
Rob

[Marvin Humphrey]

Post by Sisyphus
Looks like this is the result of a clash with the perl symbols
do_close (Perl_do_close) and do_open (Perl_do_open) which, upon
inspection of the perl source's doio.c, *do* take 2 and 7 arguments
respectively.
I don't have any advice on how to work around this particular problem.

How do you turn off the prefix-stripping for Perl symbols, so that
they have to be invoked as Perl_do_close() rather than do_close() etc?

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

[Sisyphus]

----- Original Message -----
From: "Marvin Humphrey" <***@rectangular.com>
.
.

How do you turn off the prefix-stripping for Perl symbols, so that they
have to be invoked as Perl_do_close() rather than do_close() etc?

If prefix-stripping was done (not that I know how to do it), wouldn't there
still be a problem with embed.h which defines:

#define do_close Perl_do_close
and
#define do_open Perl_do_open

Cheers,
Rob

[Jan Dubois]

Post by Marvin Humphrey

Post by Sisyphus
Looks like this is the result of a clash with the perl symbols
do_close (Perl_do_close) and do_open (Perl_do_open) which, upon
inspection of the perl source's doio.c, *do* take 2 and 7 arguments
respectively.
I don't have any advice on how to work around this particular problem.

How do you turn off the prefix-stripping for Perl symbols, so that
they have to be invoked as Perl_do_close() rather than do_close() etc?

I would try to add

#undef do_open
#undef do_close

after including the Perl headers.

Cheers,
-Jan

[Christophe Massaloux]

Post by Jan Dubois

Post by Marvin Humphrey

Post by Sisyphus
Looks like this is the result of a clash with the perl symbols
do_close (Perl_do_close) and do_open (Perl_do_open) which, upon
inspection of the perl source's doio.c, *do* take 2 and 7 arguments
respectively.
I don't have any advice on how to work around this particular problem.

How do you turn off the prefix-stripping for Perl symbols, so that
they have to be invoked as Perl_do_close() rather than do_close() etc?

I would try to add
#undef do_open
#undef do_close
after including the Perl headers.
Cheers,
-Jan

It works !!
Many thanks Jan. Your advice was the right one !
I don't know where those macros are defined in the perl headers but
undefing it is enough to work around the pb.

Best regards,

Christophe