Working with unsigned long long

[William S Fulton]

How does one marshal unsigned long long types in Perl XS?

If I have an unsigned long long number in Perl:

$num = 9234567890121111113;
print "num from perl: " . $num;

On a 32 bit Linux system I get:

num from perl: 2.34567890121111e+17

and in C code, I have:

SV* obj;
...
const char *nptr = SvPV(obj, PL_na);

then displaying nptr, I also get the scientific representation in the
string. This means I cannot use strtoull to convert to an unsigned long
long. Any portable solutions for extracting the unsigned long long number?

Thanks
William

[Sisyphus]

----- Original Message -----
From: "William S Fulton" <***@fultondesigns.co.uk>
.
.
> If I have an unsigned long long number in Perl:
>
> $num = 9234567890121111113;
> print "num from perl: " . $num;
>
> On a 32 bit Linux system I get:
>
> num from perl: 2.34567890121111e+17
>
> and in C code, I have:
>
> SV* obj;
> ...
> const char *nptr = SvPV(obj, PL_na);
>
> then displaying nptr, I also get the scientific representation in the
> string. This means I cannot use strtoull to convert to an unsigned long
> long. Any portable solutions for extracting the unsigned long long number?

But if you do:

$num = "9234567890121111113";

and then pass $num to the XSub, you should find that nptr contains the
string "9234567890121111113" which you *can* convert using strtoull.

Going the other way, unsigned long longs would also be returned from the
XSub to perl as a *string*.

I'm not sure that this achieves anything in the way of portability, however
...

Cheers,
Rob

[Torsten Schoenfeld]

On Wed, 2007-10-24 at 21:49 +0100, William S Fulton wrote:

> Any portable solutions for extracting the unsigned long long number?

The best effort I can provide is what's in Glib. They do what Sisyphus
suggests: represent big numbers as strings if necessary.

#ifdef _MSC_VER
# include <stdlib.h>
#endif

#ifdef WIN32
# ifdef _MSC_VER
# define PORTABLE_STRTOULL(str, end, base) _strtoui64 (str, end, base)
# else
# define PORTABLE_STRTOULL(str, end, base) strtoul (str, end, base)
# endif
#else
# define PORTABLE_STRTOULL(str, end, base) strtoull (str, end, base)
#endif

guint64
SvGUInt64 (SV *sv)
{
#ifdef USE_64_BIT_ALL
return SvUV (sv);
#else
return PORTABLE_STRTOULL (SvPV_nolen (sv), NULL, 10);
#endif
}

SV *
newSVGUInt64 (guint64 value)
{
#ifdef USE_64_BIT_ALL
return newSVuv (value);
#else
char string[25];
STRLEN length;
SV *sv;

/* newSVpvf doesn't seem to work correctly. */
length = sprintf(string, "%llu", value);
sv = newSVpv (string, length);

return sv;
#endif
}

This seems to work OK on most Unices, but doesn't seem to cut it on
Win32. It compiles, but according to testers, the 64 bit number tests
fail.

If anyone can offer any improvements, they'd be more than welcome.

--
Bye,
-Torsten

[m***@insulin-pumpers.org]

> On Wed, 2007-10-24 at 21:49 +0100, William S Fulton wrote:
>
> > Any portable solutions for extracting the unsigned long long number?
>
> The best effort I can provide is what's in Glib. They do what
> Sisyphus suggests: represent big numbers as strings if necessary.

You can represent them as vectors. If you only want to do +- >=<
stuff, then see the C and pureperl equiv routines in
NetAddr::IP::Util that handle 128 bit numbers. Otherwise you are
stuck with Math::BigInt

Michael
>
> #ifdef _MSC_VER
> # include <stdlib.h>
> #endif
>
> #ifdef WIN32
> # ifdef _MSC_VER
> # define PORTABLE_STRTOULL(str, end, base) _strtoui64 (str, end,
> # base)
> # else
> # define PORTABLE_STRTOULL(str, end, base) strtoul (str, end, base)
> # endif
> #else
> # define PORTABLE_STRTOULL(str, end, base) strtoull (str, end, base)
> #endif
>
> guint64
> SvGUInt64 (SV *sv)
> {
> #ifdef USE_64_BIT_ALL
> return SvUV (sv);
> #else
> return PORTABLE_STRTOULL (SvPV_nolen (sv), NULL, 10);
> #endif
> }
>
> SV *
> newSVGUInt64 (guint64 value)
> {
> #ifdef USE_64_BIT_ALL
> return newSVuv (value);
> #else
> char string[25];
> STRLEN length;
> SV *sv;
>
> /* newSVpvf doesn't seem to work correctly. */
> length = sprintf(string, "%llu", value);
> sv = newSVpv (string, length);
>
> return sv;
> #endif
> }
>
> This seems to work OK on most Unices, but doesn't seem to cut it on
> Win32. It compiles, but according to testers, the 64 bit number tests
> fail.
>
> If anyone can offer any improvements, they'd be more than welcome.
>
> --
> Bye,
> -Torsten

[Sisyphus]

----- Original Message -----
From: "Torsten Schoenfeld" <***@gmx.de>
.
.
>
> #ifdef _MSC_VER
> # include <stdlib.h>
> #endif
>
> #ifdef WIN32
> # ifdef _MSC_VER
> # define PORTABLE_STRTOULL(str, end, base) _strtoui64 (str, end, base)
> # else
> # define PORTABLE_STRTOULL(str, end, base) strtoul (str, end, base)
> # endif
> #else
> # define PORTABLE_STRTOULL(str, end, base) strtoull (str, end, base)
> #endif
>
> guint64
> SvGUInt64 (SV *sv)
> {
> #ifdef USE_64_BIT_ALL
> return SvUV (sv);
> #else
> return PORTABLE_STRTOULL (SvPV_nolen (sv), NULL, 10);
> #endif
> }
>
> SV *
> newSVGUInt64 (guint64 value)
> {
> #ifdef USE_64_BIT_ALL
> return newSVuv (value);
> #else
> char string[25];
> STRLEN length;
> SV *sv;
>
> /* newSVpvf doesn't seem to work correctly. */
> length = sprintf(string, "%llu", value);
> sv = newSVpv (string, length);
>
> return sv;
> #endif
> }
>
> This seems to work OK on most Unices, but doesn't seem to cut it on
> Win32.

It won't compile for me (on either linux or win32). I tried it as an
Inline::C script:

---------------------------------------------
use warnings;
use Inline C => Config =>
BUILD_NOISY => 1;

use Inline C => <<'EOC';
#ifdef _MSC_VER
# include <stdlib.h>
#endif

#ifdef WIN32
# ifdef _MSC_VER
# define PORTABLE_STRTOULL(str, end, base) _strtoui64 (str, end, base)
# else
# define PORTABLE_STRTOULL(str, end, base) strtoul (str, end, base)
# endif
#else
# define PORTABLE_STRTOULL(str, end, base) strtoull (str, end, base)
#endif

guint64
SvGUInt64 (SV *sv)
{
#ifdef USE_64_BIT_ALL
return SvUV (sv);
#else
return PORTABLE_STRTOULL (SvPV_nolen (sv), NULL, 10);
#endif
}

SV *
newSVGUInt64 (guint64 value)
{
#ifdef USE_64_BIT_ALL
return newSVuv (value);
#else
char string[25];
STRLEN length;
SV *sv;

/* newSVpvf doesn't seem to work correctly. */
length = sprintf(string, "%llu", value);
sv = newSVpv (string, length);

return sv;
#endif
}

EOC

print "Compiled\n";
---------------------------------------------

During compilation I get the errors:

_64bit_pl_498e.xs:20: error: syntax error before "SvGUInt64"
_64bit_pl_498e.xs:30: error: syntax error before "value"
_64bit_pl_498e.xs: In function `newSVGUInt64':
_64bit_pl_498e.xs:40: error: `value' undeclared (first use in this function)
_64bit_pl_498e.xs:40: error: (Each undeclared identifier is reported only
once
_64bit_pl_498e.xs:40: error: for each function it appears in.)

I believe the errors arise because the 'guint64' type is uknown. Is there a
missing header ?

I would think we'll also need a typemap that tells perl how to deal with the
guint64 type.

(I have a sneaking suspicion that I've missed something here :-)

As regards runtime errors, something like "%llu" will not be interpreted
correctly on Windows 2000 and earlier - you'll want to use "%I64u", "%I64x"
and "%I64d". (The same possibly applies to Windows XP - but I haven't
checked.) On my Vista 64 box I find that "%llu", "%llx", and "%lld" *are*
interpreted correctly (as also are "%I64u", "%I64x" and "%I64d"). Apparently
theWindows C runtime has been modified for Vista 64 (and probably Vista 32
as well - again I haven't checked).

Cheers,
Rob

[Muppet]

On Oct 26, 2007, at 9:33 AM, Sisyphus wrote:

> I believe the errors arise because the 'guint64' type is uknown. Is
> there a
> missing header ?

guint64 is part of GLib, since Torsten posted code from the Glib perl
module. You'll want to get uint64_t, wherever Microsoft decided to
define that (as i recall, it is *not* in the C99 standard's
inttypes.h or stdint.h headers...).


> As regards runtime errors, something like "%llu" will not be
> interpreted correctly on Windows 2000 and earlier - you'll want to
> use "%I64u", "%I64x" and "%I64d".

And those, of course, are not portable to other compilers. I've seen
the perl internals use macros to define the proper format specifiers,
and use them like so:

printf ("foo " UINT64_FORMAT "\n", some_uint64_val);


--
It's all very complicated and would take a scientist to explain it.
-- MST3K

[Torsten Schoenfeld]

On Fri, 2007-10-26 at 23:33 +1000, Sisyphus wrote:

> During compilation I get the errors:
>
> _64bit_pl_498e.xs:20: error: syntax error before "SvGUInt64"
> _64bit_pl_498e.xs:30: error: syntax error before "value"
> _64bit_pl_498e.xs: In function `newSVGUInt64':
> _64bit_pl_498e.xs:40: error: `value' undeclared (first use in this function)
> _64bit_pl_498e.xs:40: error: (Each undeclared identifier is reported only
> once
> _64bit_pl_498e.xs:40: error: for each function it appears in.)
>
> I believe the errors arise because the 'guint64' type is uknown. Is there a
> missing header ?
>
> I would think we'll also need a typemap that tells perl how to deal with the
> guint64 type.

As muppet said, these errors arise because I simply copied the code from
the Glib module. The guint64 type comes from the glib library, the
typemaps are supplied by the Glib module.

> As regards runtime errors, something like "%llu" will not be interpreted
> correctly on Windows 2000 and earlier - you'll want to use "%I64u", "%I64x"
> and "%I64d". (The same possibly applies to Windows XP - but I haven't
> checked.) On my Vista 64 box I find that "%llu", "%llx", and "%lld" *are*
> interpreted correctly (as also are "%I64u", "%I64x" and "%I64d"). Apparently
> theWindows C runtime has been modified for Vista 64 (and probably Vista 32
> as well - again I haven't checked).

Ugh, OK. So we need platform-specific format specifiers, as well.
Thanks for the information.

--
Bye,
-Torsten

[William S Fulton]

Torsten Schoenfeld wrote:
> On Wed, 2007-10-24 at 21:49 +0100, William S Fulton wrote:
>
>
>> Any portable solutions for extracting the unsigned long long number?
>>
>
> The best effort I can provide is what's in Glib. They do what Sisyphus
> suggests: represent big numbers as strings if necessary.
>
> #ifdef _MSC_VER
> # include <stdlib.h>
> #endif
>
> #ifdef WIN32
> # ifdef _MSC_VER
> # define PORTABLE_STRTOULL(str, end, base) _strtoui64 (str, end, base)
> # else
> # define PORTABLE_STRTOULL(str, end, base) strtoul (str, end, base)
> # endif
> #else
> # define PORTABLE_STRTOULL(str, end, base) strtoull (str, end, base)
> #endif
>
> guint64
> SvGUInt64 (SV *sv)
> {
> #ifdef USE_64_BIT_ALL
> return SvUV (sv);
> #else
> return PORTABLE_STRTOULL (SvPV_nolen (sv), NULL, 10);
> #endif
> }
>
Many thanks for all the postings. I think we'll just use a string in the
Perl script as even this PORTABLE_STRTOULL cannot parse the value when
stored in scientific notation.

William

[Nicholas Clark]

On Wed, Oct 24, 2007 at 09:49:26PM +0100, William S Fulton wrote:

> const char *nptr = SvPV(obj, PL_na);

Not the answer to the question you asked, but could I suggest re-writing that
as

const char *nptr = SvPV_nolen(obj);

PL_na is less efficient - for non-gcc builds it uses a global variable for
non-threaded builds, and access to thread-local storage for threaded builds.

Nicholas Clark