Re: Calculate the precision of a floating point number (ie: the number of decimal places)

In article <4727ee83$0$20603$4c368faf@xxxxxxxxxxxxxx>, lshaw-usenet@xxxxxxxxxxxxx says...
Ed Prochak wrote:
On Oct 30, 10:21 am, Lori <steve.lori...@xxxxxxxxx> wrote:
Hi all

I'm trying to write a reliable and efficient library function that
will calculate the precision of an input number (floating point).
IOW number of significant digits?

First, can you tell me how many significant digits exist in these
numbers
0.01e-21
0.00
0.000000

How are the last two going to look any different from each other in
floating point?

- Logan
Seems that it is simple to calculate the NOSD for a number input as a string, but unrealistic for a float. Floats of a
particular size are usually stored/represented internally with approximately the same number of significant digits.

Mike
.

Relevant Pages

  • Re: Floating-point arithmetic in CL
    ... When pocket calculators were introduced in French college, ... has a decimal point, but it's not a floating point, it's a fixed ... or 4 significant digits. ... The point is that the 3 significant digits of a slide rule are enough ...
    (comp.lang.lisp)
  • Re: Bug in Excel 2007
    ... number of significant digits (varies depending on the size of the data ... which fits with the double data type in C. ... nor with the IEEE standard for floating point. ... The bug is from the FPU, excel just converts its own algorithm to ...
    (microsoft.public.excel)
  • Re: how do I get more numbers past the decimal?
    ... A floating point number is stored on a 32 bit machine using the 24 bit for the mantissa and its sign and 8 bits for the exponent and its sign. ... You are showing 12 significant digits. ... A float value can be exactly represented by up to 9 decimal digits. ... this is only accurate for binary32 format. ...
    (comp.lang.php)
  • Re: Why do simple floating point calculations work ?
    ... > Skybuck Flying wrote: ... Suppose we define cell1 as: ... But because CellWidth is a floating point and position is a floating point. ...
    (alt.comp.lang.borland-delphi)
  • Re: how do I get more numbers past the decimal?
    ... A floating point number is stored on a 32 bit machine using the 24 bit for the mantissa and its sign and 8 bits for the exponent and its sign. ... You are showing 12 significant digits. ... A float value can be exactly represented by up to 9 decimal digits. ...
    (comp.lang.php)