float limits

From: ziller (ziller_at_gmail.com)
Date: 08/26/04 

Date: 25 Aug 2004 17:06:31 -0700

Why is it that FLT_DIG (from <float.h>) is 6 while DBL_DIB is 15?

Doing the math, the mantissa for floats is 24 bits = 2^24-1 max value
= 16,777,215.0f. Anything 8-digit odd # greater than that will be
rounded off.
For doubles, the mantissa is 53 bits = 2^53-1 max value =
9,007,199,254,740,991.0l (that's an L). So 16 digit odd numbers
greater than that will be rounded off. To get the actual precision we
take log(base 10) of those numbers and get 7.22 and 15.95
respectively.

...floats have greater than 7 digits precision and doubles only
greater than 15 digits. So how does MS guarantee no rounding errors
for 15 digit doubles yet 6 digit floats (if I understand correctly,
the last digit of precision must be used to round off the number...the
numbers are not just truncated at 7 & 15 digits...)

Anything I'm missing for the doubles case? It looks like they should
be guaranteeing 14 digits.

Relevant Pages

  • Re: Rounding errors
    ... Take for example 10 random single digit numbers. ... if you take a set of large precision random numbers between ... The rounding to 2 digits will restore the full count 500,500. ... But it is not the _roundeing_ that is wrong, it is the truncation to ...
    (comp.lang.cobol)
  • Re: Another question about big floating point
    ... I've been programming that bignum floating point thing I've talked ... We just lose a whole 32-bit dword "digit" from the number's ... effectively only 32 bits of precision)? ... annoyances of the hex float historically used by S/ ...
    (comp.programming)
  • =?Utf-8?Q?Re:_I_don=C2=B4t_view_the_value_right?= =?Utf-8?Q?_after_six_decimal_place_in_Exce
    ... accuracy--probably IEEE quadruple precision, which is far more than the IEEE ... The 16th digit in 2179,0089519999972104171 is a 7, which rounds up to give ... "David Biddulph" wrote: ...
    (microsoft.public.excel.misc)
  • range coder
    ... i'm fiddling around to implement a range coder as a backend to an acb ... too much precision to enable decoding. ... of the low register (he uses bit 31 to handle the underflow digit ... if the first digit (when i say digit, ...
    (comp.compression)
  • Re: long double in gcc implementations
    ... some of Mr. Navia's built-in math functions, ... char Pause; ... the full 100 digit precision claimed by qfloat. ...
    (comp.lang.c)