Re: Calculate the precision of a floating point number (ie: the number of decimal places)

From: CBFalconer <cbfalconer@xxxxxxxxx>
Date: Tue, 30 Oct 2007 16:53:37 -0500

Ed Prochak wrote:

Lori <steve.lori...@xxxxxxxxx> wrote:

I'm trying to write a reliable and efficient library function that
will calculate the precision of an input number (floating point).

IOW number of significant digits?

First, can you tell me how many significant digits exist in these
numbers
0.01e-21
0.00
0.000000

Yes. Just count the digits typed. The answers are 3, 3, 7.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>

--
Posted via a free Usenet account from http://www.teranews.com

.

References:
- Calculate the precision of a floating point number (ie: the number of decimal places)
  - From: Lori
- Re: Calculate the precision of a floating point number (ie: the number of decimal places)
  - From: Ed Prochak

Prev by Date: Re: Does the order matter to add a sequence of floating numbers
Next by Date: Red Black Guaruntees
Previous by thread: Re: Calculate the precision of a floating point number (ie: the number of decimal places)
Next by thread: Re: Calculate the precision of a floating point number (ie: the number of decimal places)
Index(es):
- Date
- Thread

Relevant Pages

Re: This calculation is just wrong / computer cant count!
... This is because of how 64-bit floating point arithmetic works. ... Any finite precision arithmetic, whether it is ... significant digits yourself. ... So we have units of inches, feet, ...
(microsoft.public.vc.mfc)
Re: Very Simple Math Error
... This is just how floating point arithmetic works on computers. ... Usually "double precision" gives ... the first 5 significant digits or so and round off the rest. ...
(comp.soft-sys.matlab)
Re: Calculate the precision of a floating point number (ie: the number of decimal places)
... will calculate the precision of an input number (floating point). ... IOW number of significant digits? ... can you tell me how many significant digits exist in these ... Seems that it is simple to calculate the NOSD for a number input as a string, but unrealistic for a float. ...
(comp.programming)
Re: (* 2.4 3) => 7.2000003 WTF?! Lets Fix Lisp! Noob Programming Challenge
... compared to the precision of the floating point numbers used ... math operations using a small number of significant digits compared ... As for reducing fractions, what I was seeking was a way to display ...
(comp.lang.lisp)
Re: Calculate the precision of a floating point number (ie: the number of decimal places)
... will calculate the precision of an input number (floating point). ... IOW number of significant digits? ... can you tell me how many significant digits exist in these ...
(comp.programming)