Re: storing an integer in a double precision



On 2011-05-05 17:50:38 -0300, Lynn McGuire said:

On 5/5/2011 12:48 PM, dpb wrote:
On 5/5/2011 12:35 PM, Lynn McGuire wrote:
...

But, do double precision variables actually store 32 bit integers
that were converted but without roundoff ? So roundoff only comes
into play for whole numbers greater than 52 bits ?
...

Well, yes, that's what the mantissa bits are for... :)

Why wouldn't they--I'm thinking there must be something behind the question, here, Lynn.

--

Just a lack of understanding of conversion between integers and
double precision on my part. Back when we were primarily single
precision, we had to be very careful, using "ii = xx + 0.1" to
ensure that we did not loose the whole number value of a
incomplete conversion of 12,000 which was stored in the float
as 11.9999.

I have noticed lately that I have gotten lazy since we converted
the entire program to double precision and don't add the 0.1
conversion factor anymore as we don't seem to need it anymore.

Thanks,
Lynn

Use NINT. NINT beats INT(x+0.5) for being clear as to intent.

Finite representation issues can go either way over time.
The 10 * 0.1 is only approximately 1 issue. You might see
a single precision 0.1 slightly small and a double precison
one slightly large or whatever fate deals out today.

.