Re: variable value rounding...
 From: Luqman <luqman_ngs@xxxxxxx>
 Date: Tue, 12 Dec 2006 13:45:22 +0100
Am Mon, 11 Dec 2006 10:40:37 0800 schrieb Mark Janssen:
Can anybody help me understand how this expression is executed such
that resulting value contains a 2 in the end instead of 1.
How can I avoid this?
% puts [expr 2.0 + 0.23385937500000001] 2.2338593750000002
Thanks,Short answer, you can't.Longer answer, the reason that you can't avoid this is that floating
point numbers cannot always be represented exactly in binary leading to
roundoff error. [http://en.wikipedia.org/wiki/Floating_point] this
makes precise calculations using floating point on computers a tricky
business. If you really need this precision you can change to using
ints (Tcl 8.5 bigints for example)
Mark
Another nice writeup on the subject can be found at
[http://wiki.tcl.tk/11969]
Thanks Mark for your helpful reply. I am new to tcl and can't solve this
problem:
%set x 2.0
%set y 0.23385937500000001
%puts [expr $x +$y]
How would I typecast $x and $y or $z to bigfloat/double/bigints? My goal
is to have a result that gives me the precision required to sum up the
above mentioned $y (with 17 number after point) and $y without loss.
Is it feacible?
Thanks,

Luqman
.
 FollowUps:
 Re: variable value rounding...
 From: Arjen Markus
 Re: variable value rounding...
 References:
 variable value rounding...
 From: Luqman
 Re: variable value rounding...
 From: Mark Janssen
 Re: variable value rounding...
 From: Mark Janssen
 variable value rounding...
 Prev by Date: Re: TCL does nit exist ;)
 Next by Date: Re: variable value rounding...
 Previous by thread: Re: variable value rounding...
 Next by thread: Re: variable value rounding...
 Index(es):
Relevant Pages
