Re: Can someone please explain - real numbers

John Dough <nobody@xxxxxxxxxxxxxxx> wrote in
news:jat3l2lpfacc1i4fknftrhncqtjrto38lt@xxxxxxx:

On Wed, 08 Nov 2006 06:45:45 +0000, Paul W <noemail@xxxxxxxxx> wrote:

I always thought a computer would store something like 1/3
(0.33333...) simply as 0.33333... for as many decimal places it could
and then stopped. Is that not the case then?

Yes, that's exactly correct.
If you store 1/3 into an extended variable, it will store 0.3
(repeated down to 18 decimal places).

The only exception is if the last digit has to be rounded up, as in
the case of 2/3. It will store "0.666666666666666667". But this
isn't anything surprising, because it's something we all learned back
in grade 3.

Now there's one important thing to remember here. If you need to
store a fraction directly, then it's perfectly okay to check for
equality (as in your original example). A non-repeating fraction
will also work fine. But if you store a repeating fraction as a
result of a calculation, then you have to do one extra step before you
can accurately check it for equality.

Note that, for example, the simple decimal fraction 0.1 is the recurring
binary fraction 0.0001100110011001100110011001100110 (0110 recurring) and
cannot therefore be stored precisely as a single/double/extended/real.
.

Relevant Pages

  • Applesoft FP storage
    ... How many bytes are used by Applesoft to store the above numbers? ... with the 6 unused bits in the fraction bytes, ... that FP in Applesoft is including precision that I don't ...
    (comp.sys.apple2.programmer)
  • Re: is there a reason why to make mix numbers improper when adding?
    ... > Is there a reason to store a fraction as a whole number and a fractional ... >> It seems when subtracting and adding, ... > same as adding or subtracting two improper fractions. ...
    (comp.lang.cpp)
  • Re: Weird problem ( 3 * 31.9 = 95.69999999999999)
    ... >>to store the fractional part of a calculation. ... >>you'll have 4 bits left for the fraction: ... > That's not floating point, ... > I suggest that you look up hoe IEEE Doubles are formatted internally; ...
    (comp.lang.javascript)
  • Re: Weird problem ( 3 * 31.9 = 95.69999999999999)
    ... >to store the fractional part of a calculation. ... >you'll have 4 bits left for the fraction: ... That's not floating point, it's fixed point. ... you can find it by way of one of the links in the newsgroup FAQ. ...
    (comp.lang.javascript)
  • Re: Totally Official SDC June teaser
    ... You mean the ratio of Greek to Japanese. ... No, as the question's author, I meant equality. ... fraction are equal. ...
    (alt.usage.english)