Re: storing an integer in a double precision

On 5/5/2011 12:08 PM, Richard Maine wrote:
Lynn McGuire<lmc@xxxxxxxxxx> wrote:

Is there a rule of thumb for the biggest integer that
I can store in a double precision variable without losing
the integer value due to round off ? BTW, I use a F77
compiler.

No particular "rule of thumb". Just look at the particular
representations. It doesn't have much to do with the version of Fortran,
but with the physical representation used for double precision (and, to
a lesser extent for integer, except that you won't run into any machines
that store integers in other than binary). Heck, it barely even has to
do with Fortran. (A little, but barely; the little has to do with how
Fortran compilers could select from different physical representations
supported by the hardware).

Look at how many bits are in the mantissa of the representation for
double. That's about how large an integer you could store without
roundoff. If you want the exact number, you have to look more carefully
and consider things like hidden bits (and on old IBM mainframes,
exponent radix). But for a rough approximation, just look at the number
of bits in the mantissa.

Most compilers these days use IEEE double, which has, if I recall
correctly, 53 bits in the mantissa. So your answer would be somewhere
around 2**53.

But, do double precision variables actually store 32 bit integers
that were converted but without roundoff ? So roundoff only comes
into play for whole numbers greater than 52 bits ?

Thanks,
Lynn
.

Relevant Pages

  • Re: storing an integer in a double precision
    ... I can store in a double precision variable without losing ... It doesn't have much to do with the version of Fortran, ... but with the physical representation used for double precision (and, ... of bits in the mantissa. ...
    (comp.lang.fortran)
  • Re: storing an integer in a double precision
    ... Most compilers these days use IEEE double, which has, if I recall ... do double precision variables actually store 32 bit integers ... that were converted but without roundoff? ...
    (comp.lang.fortran)
  • Re: Applesoft: find address that variable is stored at?
    ... anyone have a better explanation of the 5 byte representation? ... format works like this: ... the mantissa is stored in the other four bytes. ... Since this is always known to be the case, we don't have to actually store these bits anywhere. ...
    (comp.sys.apple2.programmer)
  • Re: Rounnding Issue
    ... In SQL Server FLOAT can use up to 53 bits to store the mantissa. ... Depending on how you calculator is implemented you can have different results. ...
    (microsoft.public.sqlserver.programming)
  • Re: Floating Point Data Compression?
    ... Well, to reduce the number of bits you need to store, you could throw away ... some mantissa bits. ... That's probably the simplest form of compression, ...
    (comp.dsp)