Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

Hello everybody -

I recently resuscitated CUDA::Minimal, a module that provides basic memory
allocation and memory transfer functionality for CUDA (i.e. video card)
parallel programming. Note, you must write your own CUDA kernels
separately; I recommend writing them in XS wrapper code, using
ExtUtils::nvcc and can answer questions if you're interested.

When using ExtUtils::nvcc, your code gets compiled through a C++ compiler.
I seem to have struck a problem with how XS handles its dVAR and dXSARGS
declarations in the Boot section and in other sections. The applicable
error can be found here:
https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code
that leads to this error (generated from the XS code on my machine) is
given in this gist: https://gist.github.com/run4flat/4974702.

The problem is a linkage conflict in the declaration for Perl__notused on
lines 163 <https://gist.github.com/run4flat/4974702#file-minimal-c-L163>and
329 <https://gist.github.com/run4flat/4974702#file-minimal-c-L374>, which I
presume is embedded in dVAR or dXSARGS. However, I'm left scratching my
head as to why the linkage would differ between these two. Based on my
understanding of how extern "C"
works<http://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage>,
there is no difference that I can see. That is, there is no difference in
the linkage specification for the two, because the extern "C" on line 369
does not have curly braces and therefore only effects the declaration of
the function definition. It should not impact the declarations of any
variables.

Anybody have any idea what's causing this linkage specification problem,
and how I might fix it?

Thanks!
David

[bulk88]

Post by David Mertens
Hello everybody -
I recently resuscitated CUDA::Minimal, a module that provides basic
memory allocation and memory transfer functionality for CUDA (i.e.
video card) parallel programming. Note, you must write your own CUDA
kernels separately; I recommend writing them in XS wrapper code, using
ExtUtils::nvcc and can answer questions if you're interested.
When using ExtUtils::nvcc, your code gets compiled through a C++
compiler. I seem to have struck a problem with how XS handles its dVAR
and dXSARGS declarations in the Boot section and in other sections.
https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C
code that leads to this error (generated from the XS code on my
machine) is given in this gist: https://gist.github.com/run4flat/4974702.
The problem is a linkage conflict in the declaration for Perl__notused
on lines 163
<https://gist.github.com/run4flat/4974702#file-minimal-c-L163> and 329
<https://gist.github.com/run4flat/4974702#file-minimal-c-L374>, which
I presume is embedded in dVAR or dXSARGS. However, I'm left scratching
my head as to why the linkage would differ between these two. Based on
my understanding of how extern "C" works
<http://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage>,
there is no difference that I can see. That is, there is no difference
in the linkage specification for the two, because the extern "C" on
line 369 does not have curly braces and therefore only effects the
declaration of the function definition. It should not impact the
declarations of any variables.
Anybody have any idea what's causing this linkage specification
problem, and how I might fix it?
Thanks!
David

I think dVAR becomes dNOOP which is
http://perl5.git.perl.org/perl.git/blob/f6fe275c937ceb508cdcfd033ce162e1fa01989e:/perl.h#l354.
Note a C++ and a C version of that macro. Can you post the preprocessed
version of the .c file or have you looked at the preprocessed version
for clues as to what happened?

[David Mertens]

Sorry, meant to reply-all.

David

---------- Forwarded message ----------
From: David Mertens <***@gmail.com>
Date: Mon, Feb 18, 2013 at 12:51 AM
Subject: Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled
using nvcc
To: bulk88 <***@hotmail.com>


bulk88 -

I've updated the original gist with a new file that includes the direct
C(++) code as preprocessed by nvcc. The two look identical to me. Should I
try to bump-up the verbosity of the compiler?

David

Post by David Mertens
**
Hello everybody -
I recently resuscitated CUDA::Minimal, a module that provides basic
memory allocation and memory transfer functionality for CUDA (i.e. video
card) parallel programming. Note, you must write your own CUDA kernels
separately; I recommend writing them in XS wrapper code, using
ExtUtils::nvcc and can answer questions if you're interested.
When using ExtUtils::nvcc, your code gets compiled through a C++
compiler. I seem to have struck a problem with how XS handles its dVAR and
dXSARGS declarations in the Boot section and in other sections. The
https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code
that leads to this error (generated from the XS code on my machine) is
given in this gist: https://gist.github.com/run4flat/4974702.
The problem is a linkage conflict in the declaration for Perl__notused on
lines 163 <https://gist.github.com/run4flat/4974702#file-minimal-c-L163>and
329 <https://gist.github.com/run4flat/4974702#file-minimal-c-L374>, which
I presume is embedded in dVAR or dXSARGS. However, I'm left scratching my
head as to why the linkage would differ between these two. Based on my
understanding of how extern "C" works<http://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage>,
there is no difference that I can see. That is, there is no difference in
the linkage specification for the two, because the extern "C" on line 369
does not have curly braces and therefore only effects the declaration of
the function definition. It should not impact the declarations of any
variables.
Anybody have any idea what's causing this linkage specification problem,
and how I might fix it?
Thanks!
David
I think dVAR becomes dNOOP which is
http://perl5.git.perl.org/perl.git/blob/f6fe275c937ceb508cdcfd033ce162e1fa01989e:/perl.h#l354.
Note a C++ and a C version of that macro. Can you post the preprocessed
version of the .c file or have you looked at the preprocessed version for
clues as to what happened?

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan

[David Mertens]

Sorry for the rapid-fire. I'm up late and the ideas are only trickling out.

It appears that the code doesn't think __cplusplus is defined. That is odd.
nvcc claims (or, it used to) that it always runs the code through a C++
compiler at the end of the day, so why it wouldn't have __cpluplus defined
is beyond me.

David

Post by David Mertens
Sorry, meant to reply-all.
David
---------- Forwarded message ----------
Date: Mon, Feb 18, 2013 at 12:51 AM
Subject: Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled
using nvcc
bulk88 -
I've updated the original gist with a new file that includes the direct
C(++) code as preprocessed by nvcc. The two look identical to me. Should I
try to bump-up the verbosity of the compiler?
David

Post by David Mertens
**
Hello everybody -
I recently resuscitated CUDA::Minimal, a module that provides basic
memory allocation and memory transfer functionality for CUDA (i.e. video
card) parallel programming. Note, you must write your own CUDA kernels
separately; I recommend writing them in XS wrapper code, using
ExtUtils::nvcc and can answer questions if you're interested.
When using ExtUtils::nvcc, your code gets compiled through a C++
compiler. I seem to have struck a problem with how XS handles its dVAR and
dXSARGS declarations in the Boot section and in other sections. The
https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C
code that leads to this error (generated from the XS code on my machine) is
given in this gist: https://gist.github.com/run4flat/4974702.
The problem is a linkage conflict in the declaration for Perl__notused
on lines 163<https://gist.github.com/run4flat/4974702#file-minimal-c-L163>and
329 <https://gist.github.com/run4flat/4974702#file-minimal-c-L374>,
which I presume is embedded in dVAR or dXSARGS. However, I'm left
scratching my head as to why the linkage would differ between these two.
Based on my understanding of how extern "C" works<http://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage>,
there is no difference that I can see. That is, there is no difference in
the linkage specification for the two, because the extern "C" on line 369
does not have curly braces and therefore only effects the declaration of
the function definition. It should not impact the declarations of any
variables.
Anybody have any idea what's causing this linkage specification problem,
and how I might fix it?
Thanks!
David
I think dVAR becomes dNOOP which is
http://perl5.git.perl.org/perl.git/blob/f6fe275c937ceb508cdcfd033ce162e1fa01989e:/perl.h#l354.
Note a C++ and a C version of that macro. Can you post the preprocessed
version of the .c file or have you looked at the preprocessed version for
clues as to what happened?

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." -- Brian Kernighan

[bulk88]

Post by David Mertens
Sorry for the rapid-fire. I'm up late and the ideas are only trickling out.
It appears that the code doesn't think __cplusplus is defined. That is
odd. nvcc claims (or, it used to) that it always runs the code through
a C++ compiler at the end of the day, so why it wouldn't have
__cpluplus defined is beyond me.
David

Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
Read the compilers manual or google it. Also your XS module needs to use
PERL_NO_GET_CONTEXT for efficiency.

your file is called "Minimal.c", per
http://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdf
your file will be compiled in regular C mode. Try a (untested)

#ifndef __cplusplus
#error c++ is not defined
#endif

for debugging.

[David Mertens]

bulk88 -

nvcc knows about C, C++, and CUDA-C. I do not recall if it differentiates
between C and C++, though. I know that I correctly specify that Minimal.c
should be compiled as CUDA-C, but I'm not sure if I told it to do that in
the preprocessor command that produced the code I included in the above
gist. I will have to check when I get home tonight, which will be in 8
hours or so.

As for the #ifndef error idea for debugging, that's perfect. I'll give it a
run and see what it produces.

David

Post by bulk88

Post by David Mertens
Sorry for the rapid-fire. I'm up late and the ideas are only trickling out.
It appears that the code doesn't think __cplusplus is defined. That is
odd. nvcc claims (or, it used to) that it always runs the code through a
C++ compiler at the end of the day, so why it wouldn't have __cpluplus
defined is beyond me.
David

Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
Read the compilers manual or google it. Also your XS module needs to use
PERL_NO_GET_CONTEXT for efficiency.
your file is called "Minimal.c", per http://sbel.wisc.edu/Courses/**
ME964/2008/Documents/**nvccCompilerInfo.pdf<http://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdf>your file will be compiled in regular C mode. Try a (untested)
#ifndef __cplusplus #error c++ is not defined
#endif
for debugging.

[David Mertens]

Sorry for not getting back to this as quickly as I had hoped. My machine
with nVidia hardware is at home and I haven't had much time in the las few
evenings to play with it. This may be the case through the weekend; we
shall see.

bulk88, I was not entirely sure what PERL_NO_GET_CONTEXT did. After some
digging, I came across this article that seemed to clarify things quite a
bit, at least for me:
http://perl.active-venture.com/pod/perlguts-concurreny.html. I also did
some reading and discovered that there are ways to have thread-local data
in C, which was news to me. (I never learned the intricacies of C.)
Suddenly the behavior of dTHX; and the whole api with the _nocontext suffix
make so much more sense.

Thanks!
David

Post by bulk88

Post by David Mertens
Sorry for the rapid-fire. I'm up late and the ideas are only trickling out.
It appears that the code doesn't think __cplusplus is defined. That is
odd. nvcc claims (or, it used to) that it always runs the code through a
C++ compiler at the end of the day, so why it wouldn't have __cpluplus
defined is beyond me.
David

Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
Read the compilers manual or google it. Also your XS module needs to use
PERL_NO_GET_CONTEXT for efficiency.
your file is called "Minimal.c", per http://sbel.wisc.edu/Courses/**
ME964/2008/Documents/**nvccCompilerInfo.pdf<http://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdf>your file will be compiled in regular C mode. Try a (untested)
#ifndef __cplusplus #error c++ is not defined
#endif
for debugging.

[bulk88]

Post by David Mertens
Sorry for not getting back to this as quickly as I had hoped. My
machine with nVidia hardware is at home and I haven't had much time in
the las few evenings to play with it. This may be the case through the
weekend; we shall see.
bulk88, I was not entirely sure what PERL_NO_GET_CONTEXT did. After
some digging, I came across this article that seemed to clarify things
http://perl.active-venture.com/pod/perlguts-concurreny.html. I also
did some reading and discovered that there are ways to have
thread-local data in C, which was news to me. (I never learned the
intricacies of C.) Suddenly the behavior of dTHX; and the whole api
with the _nocontext suffix make so much more sense.
Thanks!
David

PERL_NO_GET_CONTEXT removed a bazillion func calls to Perl_get_context
or in your case (since you are posix os), pthread_getspecific. This
makes your XS code much faster (20 to 100% faster). Look at the
preprocessed version of the .c file at
https://gist.github.com/run4flat/4974702 . Sadly PERL_NO_GET_CONTEXT's
biggest reason to use it (performance) isn't in perlguts. Its main
purpose it to remove the extra get context func calls. The thread local
data stuff (
http://perldoc.perl.org/perlxs.html#Safely-Storing-Static-Data-in-XS )
few modules use (they normally get_sv a package SV), but there is no
reason why you can't use it.