Re: storing an integer in a double precision
 From: glen herrmannsfeldt <gah@xxxxxxxxxxxxxxxx>
 Date: Thu, 5 May 2011 21:30:43 +0000 (UTC)
Lynn McGuire <lmc@xxxxxxxxxx> wrote:
(snip)
Just a lack of understanding of conversion between integers and
double precision on my part. Back when we were primarily single
precision, we had to be very careful, using "ii = xx + 0.1" to
ensure that we did not loose the whole number value of a
incomplete conversion of 12,000 which was stored in the float
as 11.9999.
Well, that is a different question. One hopes that converting
an integer to floating point gives the exact value, and the
conversion back also.
Other than that, there is no guarantee that one gets the exact
value, and many cases where it is unlikely or at least even
odds.
xx = 1./3.
yy = xx * 3
j = yy
Has fairly good odds of not giving 1 for j.
I have noticed lately that I have gotten lazy since we converted
the entire program to double precision and don't add the 0.1
conversion factor anymore as we don't seem to need it anymore.
Not so obvious to me. Though IEEE has some rounding requirements
that others don't, which might increase the odds.
 glen
.
 References:
 storing an integer in a double precision
 From: Lynn McGuire
 Re: storing an integer in a double precision
 From: Richard Maine
 Re: storing an integer in a double precision
 From: Lynn McGuire
 Re: storing an integer in a double precision
 From: Lynn McGuire
 storing an integer in a double precision
 Prev by Date: Re: storing an integer in a double precision
 Next by Date: Re: storing an integer in a double precision
 Previous by thread: Re: storing an integer in a double precision
 Next by thread: Re: storing an integer in a double precision
 Index(es):
Relevant Pages
