Re: Calculate the precision of a floating point number (ie: the number of decimal places)
- From: Lori <steve.lorimer@xxxxxxxxx>
- Date: Wed, 31 Oct 2007 11:33:44 -0000
On Oct 30, 7:28 pm, Ed Prochak <edproc...@xxxxxxxxx> wrote:
IOW number of significant digits?
I guess
First, can you tell me how many significant digits exist in these
numbers
0.01e-21
0.00
0.000000
???????
1. 0.01e-21 = 0.00000000000000000000001: so 23 decimal places
2. 0.00 = 0: so 0 decimal places
3. 0.000000 = 0: so 0 decimal places
To use a New hampshire proverb:
you can't get there from here
What you need is to better define your requirements.
Ed
For sure. So the requirement is this: represent the decimal number as
an integer and a "format indicator", which is '0' - '9' for a positive
number with 0 to 9 decimal places respectively, or 'A' to 'J' for a
negative number with 0 to 9 decimal places respectively
Therefore: 34.125 would be 334125, and -10.0005 would be D100005
So my function is to take any "double" input, calculate the number of
dec places / precision / sig digits (not sure of correct terminology,
of if indeed there is any difference there), and return an integer.
ie: for
input 34.125 output 3
input -10.0005 output 4
input 0.0000 output 0
.
- Follow-Ups:
- References:
- Prev by Date: Re: SortMerge avoids thrashing (was: Bailey's "two pass" FFT algorithm question)
- Next by Date: Re: Does the order matter to add a sequence of floating numbers
- Previous by thread: Re: Calculate the precision of a floating point number (ie: the number of decimal places)
- Next by thread: Re: Calculate the precision of a floating point number (ie: the number of decimal places)
- Index(es):