Re: storing an integer in a double precision

On 5/5/2011 12:48 PM, dpb wrote:
On 5/5/2011 12:35 PM, Lynn McGuire wrote:
...

But, do double precision variables actually store 32 bit integers
that were converted but without roundoff ? So roundoff only comes
into play for whole numbers greater than 52 bits ?
...

Well, yes, that's what the mantissa bits are for... :)

Why wouldn't they--I'm thinking there must be something behind the question, here, Lynn.

--

Just a lack of understanding of conversion between integers and
double precision on my part. Back when we were primarily single
precision, we had to be very careful, using "ii = xx + 0.1" to
ensure that we did not loose the whole number value of a
incomplete conversion of 12,000 which was stored in the float
as 11.9999.

I have noticed lately that I have gotten lazy since we converted
the entire program to double precision and don't add the 0.1
conversion factor anymore as we don't seem to need it anymore.

Thanks,
Lynn
.

Relevant Pages

  • Re: storing an integer in a double precision
    ... that were converted but without roundoff? ... double precision on my part. ... conversion factor anymore as we don't seem to need it anymore. ...
    (comp.lang.fortran)
  • Re: storing an integer in a double precision
    ... double precision on my part. ... incomplete conversion of 12,000 which was stored in the float ... there is no guarantee that one gets the exact ... conversion factor anymore as we don't seem to need it anymore. ...
    (comp.lang.fortran)
  • Re: storing an integer in a double precision
    ... double precision on my part. ... conversion factor anymore as we don't seem to need it anymore. ... allegedly because of some bad experience in a compiler from ...
    (comp.lang.fortran)
  • Re: storing an integer in a double precision
    ... On 5/5/11 3:50 PM, Lynn McGuire wrote: ... that were converted but without roundoff? ... into play for whole numbers greater than 52 bits? ... You must have worked only with positive numbers, as otherwise your conversion is in error. ...
    (comp.lang.fortran)
  • Re: IBM2435I on ROUND(x,-3)
    ... fraction. ... If an intermediate result is necessary, as in evaluation of an operational expression, the attributes of the intermediate result are the same as if a decimal fixed-point value of precision had appeared in place of the string. ... In other words, FIXED DECIMALwould be converted to FLOAT DECIMAL, and in the actual conversion process, any fractional part of the constant expressed in character form would influence the exponent of the converted result and the fractional digits would not actually be lost. ... ROUNDreturns the value of the real expression x rounded at a digit specified by n. ...
    (comp.lang.pl1)