Re: Simple maths question

On May 11, 5:27 am, Mark Janssen <mpc.jans...@xxxxxxxxx> wrote:
On 10 mei, 22:32, Frem <freminl...@xxxxxxxxx> wrote:



Hello,

I was demonstrating how easy Tcl is to use (or at least to learn to use)
when I was immediately flummoxed.

On a Windows XP machine I created a trivial proc:

(bin) 289 % proc subtract {a b} {
return [expr {$a - $b}]

}

(bin) 290 % subtract 3 2.2
0.7999999999999998

Eh? This was using Tcl 8.5a4. It works properly in 8.4.11. Is this a
known bug (sorry if it is)?

Thanks,
Frem.

It is working properly in 8.5 as well. The difference between 8.4 and
8.5 is that the tcl_precision global is set differently by default.
In 8.5 the two lines:

set tcl_precision 12
subtract 3 2.2

will give the same result as in 8.4.

The real reason you get this result however is that floating point
math is not precise on a computer (0.2 is a repeating binary
fraction).

People should be reminded with floating point, any fraction that
doesn't end in a '5' is imprecise. Just like with regular pen-on-paper
decimal math, 1/3 is imprecise (interestingly if you do your math in
base 3, 1/3 is not a repeating fraction and can be precisely
represented).

.

Relevant Pages