Re: Can someone please explain - real numbers

On Wed, 08 Nov 2006 06:45:45 +0000, Paul W <noemail@xxxxxxxxx> wrote:

I always thought a computer would store something like 1/3
(0.33333...) simply as 0.33333... for as many decimal places it could
and then stopped. Is that not the case then?

Yes, that's exactly correct.
If you store 1/3 into an extended variable, it will store 0.3
(repeated down to 18 decimal places).

The only exception is if the last digit has to be rounded up, as in
the case of 2/3. It will store "0.666666666666666667". But this
isn't anything surprising, because it's something we all learned back
in grade 3.

Now there's one important thing to remember here. If you need to
store a fraction directly, then it's perfectly okay to check for
equality (as in your original example). A non-repeating fraction
will also work fine. But if you store a repeating fraction as a
result of a calculation, then you have to do one extra step before you
can accurately check it for equality.

Here's what I mean:

a := 0.333333333333333333;
if a = 0.333333333333333333 then showmessage('Equality found');
This will work fine.

However...if you do it this way:
a := 1/3;
if a = 0.333333333333333333 then showmessage('Equality found');
This will NOT return true, UNLESS you do a simple FloatToStr before
you check it.

Here's the working version:
a := 1/3;
if floattostrf(a,ffnumber,18,18) = '0.333333333333333333' then
showmessage('Equality found');

And finally...

a := 1/8;
if a = 0.125 then showmessage('Equality found');
This will work fine because it's not a repeating fraction (even though
it comes from a direct calculation).


So equally 2.1 would simply be stored 2.100000... again for as many
digits as that particular number "type" allowed.

That's correct. If you use the extended type, it will store (and
accurately return) 2.100000000000000000 when you check it.

Real/single/double are not reliable because (as others have pointed
out), the results get butchered when they're typecast or "promoted" to
extended.

This isn't as complicated as some self-proclaimed experts will have
you believe. They'll try and convince you that you can't do this or
shouldn't do that, because if you do, the sun will undoubtedly
explode. As I've already demonstrated, you CAN do what you initially
wanted to do, and you can do it safely and reliably.
.