Re: Calculate the precision of a floating point number (ie: the number of decimal places)



On Oct 30, 7:21 am, Lori <steve.lori...@xxxxxxxxx> wrote:
Hi all

I'm trying to write a reliable and efficient library function that
will calculate the precision of an input number (floating point).

Please excuse my lame mathematics skills! Here is my first attempt at
a function:

int dec_places(double d)
{
int places = 0;
while (!feq(d - (int)(d + 0.5), 0.0)) // where feq returns
true if inputs differ by less than epsilon 0.0000005
{
d *= 10;
++places;
}
return places;
}

The problem I have with the above solution is that it is possible to
get caught in an infinite loop.

One solution would be to have a max iterations, and break out of the
loop if this is reached (thereby causing an error condition, and the
caller must process the return value as an invalid response), but this
is not ideal.

I have searched for answers on the web and in newsgroups to no avail.

Does anyone know of a better algorithm? Perhaps there is an open
source solution available that I am not aware of?

You cannot know the precision of a value by examination.
You need to ask the user for the precision (and hope that they know
the answer).
The user input '1' may have infinite precision (there is exactly one
item). E.g. 'How many "Empire State Buildings in New York, New York
are there?"
The user input 3.1425936535 may have 3 digits of precision because the
digits will not be repeatable on the next measurement and they are
simply reporting all the digits they see when measuring or
calculating.

.



Relevant Pages

  • Re: Calculate the precision of a floating point number (ie: the number of decimal places)
    ... I'm trying to write a reliable and efficient library function that ... will calculate the precision of an input number. ... can you tell me how many significant digits exist in these ... get caught in an infinite loop. ...
    (comp.programming)
  • Re: float bug? perl 5.8, DBI and oracle 10.2.0
    ... precision numbers in oracle, you've got 38 decimal digits to play ... and with minimal coaxing perl will handle them as ... digits from a 32 bit floating point number - I'll go out on a limb ... and hazard that one can expect 12 or so digits from a 64 bit floating ...
    (perl.dbi.users)
  • RE: float bug? perl 5.8, DBI and oracle 10.2.0
    ... I would not characterise 32-bit signed integers as giving 10 digits ... truncate and tell people you get 9 digits of precision. ... perl 5.8, DBI and oracle 10.2.0 ... Floating point values are typically stored in 64 bits or sometimes 96 ...
    (perl.dbi.users)
  • Re: float bug? perl 5.8, DBI and oracle 10.2.0
    ... I would not characterise 32-bit signed integers as giving 10 digits ... truncate and tell people you get 9 digits of precision. ... perl 5.8, DBI and oracle 10.2.0 ... Floating point values are typically stored in 64 bits or sometimes 96 ...
    (perl.dbi.users)
  • Re: 15 Significant Digits Limitation a Mistake for Spatial Informa
    ... DP does not restrict to 15 decimal digits. ... Input and output precision are more tightly linked in Excel ... Decimal data type or roll your own extended precision data types. ...
    (microsoft.public.excel)