RE: When is a string a number?

This is a very interesting problem, Martin, because most of us ordinary
Perl users don't experience these issues on a regular basis if ever.

Just one thought, maybe a bit naiive, but wouldn't it be possible to
copy all the values you've got from the database and do the extra
computations on the copies separately? It might avoid polluting the
originals before passing them off to JSON.

-Will


-----Original Message-----
From: Martin J. Evans [mailto:martin.evans@xxxxxxxxxxxx]
Sent: Wednesday 04 October 2006 15:36
To: dbi-users
Subject: Re: When is a string a number?


On Wed, 2006-10-04 at 14:00 -0500, David Nicol wrote:
On 10/4/06, Martin J. Evans <martin.evans@xxxxxxxxxxxx> wrote:

With DBI/DBD::Oracle all values read from the database
are scalars. As
everyone will know, whether something read from the
database is a string
or a number in Perl purely depends on the context it is
used in so:

internally, there are flags in the SV structure that hint
as to what conversions
have been done on the SV already, for efficiency's sake.

As I suspected.

I have to admit I don't know how the JSON module knows
what is a number
and what is a string in Perl but I see the same issue
with Data::Dumper
so I presume there must be some way to find out if a perl
scalar is a
number or a string.

It seems like the pure perl Dumper tests values with a
regex and optimizes
to numbers when the stringification matches a number template, while
the XS version checks the flags in the SV structure.

OK, I only introduced the Data::Dumper example as it seemed
very similar
but read on - I over simplified a little. However, it seems strange to
me that the pure perl Data::Dumper does something different. If I was
using Data::Dumper for data interchange and switched from the
pure perl
to the XS version (or vice versa) and then called another XS based
module with the results - they could be different.

The problem gets a lot worse for me since I do some
arithmetic on values
pulled from the database before converting them to JSON
and this is
where Perl seems to change them into numbers e.g.

I don't want to have to do ($var +0) on all the number
fields I pull
from the database (to turn them into numbers) and neither
do I want to
do a '$var .= ""' (to turn all the fields into strings).

you might have to do exactly that. "$var" will produce a
string version. Sorry.

This seems amazing - I really did not want to do this. I
guess what I'm
saying is that it would be really useful for a DBD to mark a
number as a
number so it does not start as a string and mutate to a
number when I do
arithmetic on it. Most DBDs must know if the column is a number one or
not (certainly DBD::ODBC does).

Apparently you can tell the JSON module to make everything strings:
http://search.cpan.org/~makamaka/JSON-1.07/lib/JSON.pm#AUTOCONVERT

Here was an over simplification. I moved from JSON to JSON Syck. The
JSON module is a purl perl module and does "the right thing"
for me for
all the cases I tried but I changed to JSON Syck which uses
the libsyck
library for speed. This explains the difference since JSON Syck is XS
and will know the flags in the SV structure and JSON won't.

As an aside (and probably a perl question rather than a
DBI one) does
anyone know why the type of a scalar changes when you use
it on the
right side of an assignment:

perl -MData::Dumper -le '$a="1"; print Dumper($a); $b += $a; print
Dumper($a);'
$VAR1 = '1';
$VAR1 = 1;

The has-been-evaluated-as-a-number flag got set on $a when it was
evaluated as a number; then Dumper, with both available, chose the
number format.

That's interesting to know, I'll look into that. I was rather
surprised
the right hand side of an evaluation changed the type of a variable
on the right hand side.

How does JSON and Data::Dumper know whether Perl thinks
something is a
number or a string?

inspecting the flags; except pure-perl Dumper apparently uses a
regular expression
to identify numbers. Those are guesses. The source is available for
your inspection.

I did look at it briefly but not extensively since it was not
my primary
problem.

Thanks for your insights David, they have increased my understanding
although I still feel stuck. I could move back to using the
JSON module
but I could run into problems with fields like house_name_or_number
which I'm now guessing JSON will serialise as a number when it looks
like a number and a string when not. If I stick with JSON Syck
(which is a lot faster and I need the speed), I'm
forced to do +0 on all database fields I know are a number and I don't
like this at all. I guess I'm rather surprised to hit this
issue without
seeing anyone else with it after all this time using DBI.

Martin
--
Martin J. Evans
Easysoft Limited
http://www.easysoft.com



- - - - - Appended by Scientific Atlanta, a Cisco company - - - - -
This e-mail and any attachments may contain information which is confidential, proprietary, privileged or otherwise protected by law. The information is solely intended for the named addressee (or a person responsible for delivering it to the addressee). If you are not the intended recipient of this message, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete it from your computer.

.

Relevant Pages

  • RE: When is a string a number?
    ... Perl users don't experience these issues on a regular basis if ever. ... the values computed from the DB values are required in the JSON ... When is a string a number? ... I moved from JSON to JSON Syck. ...
    (perl.dbi.users)
  • Re: When is a string a number?
    ... whether something read from the database is a string ... or a number in Perl purely depends on the context it is used in so: ... number or a string. ... I moved from JSON to JSON Syck. ...
    (perl.dbi.users)
  • Re: Prototype zickt herum
    ... es kennt sich hier jemand mit Prototype & Perl aus. ... Für die Clientseite habe ich folgende HTML Seite: ... Der 2.Parameter im Callback (hier "json") ist optional. ... Das Skript macht aber eigentlich das, ...
    (de.comp.lang.perl.cgi)
  • Re: DB access problems
    ... That would defeat the purpose of using AJAX in the first ... What you want to produce on the server is either XML or JSON, ... data structure - have a look at the JSON module on CPAN. ... The key is that it isn't just a single request, ...
    (comp.lang.perl.misc)