Re: storing an integer in a double precision
- From: glen herrmannsfeldt <gah@xxxxxxxxxxxxxxxx>
- Date: Thu, 5 May 2011 21:30:43 +0000 (UTC)
Lynn McGuire <lmc@xxxxxxxxxx> wrote:
Just a lack of understanding of conversion between integers and
double precision on my part. Back when we were primarily single
precision, we had to be very careful, using "ii = xx + 0.1" to
ensure that we did not loose the whole number value of a
incomplete conversion of 12,000 which was stored in the float
Well, that is a different question. One hopes that converting
an integer to floating point gives the exact value, and the
conversion back also.
Other than that, there is no guarantee that one gets the exact
value, and many cases where it is unlikely or at least even
xx = 1./3.
yy = xx * 3
j = yy
Has fairly good odds of not giving 1 for j.
I have noticed lately that I have gotten lazy since we converted
the entire program to double precision and don't add the 0.1
conversion factor anymore as we don't seem to need it anymore.
Not so obvious to me. Though IEEE has some rounding requirements
that others don't, which might increase the odds.
- Prev by Date: Re: storing an integer in a double precision
- Next by Date: Re: storing an integer in a double precision
- Previous by thread: Re: storing an integer in a double precision
- Next by thread: Re: storing an integer in a double precision