Re: Can someone please explain - real numbers

On Wed, 08 Nov 2006 20:51:34 GMT, Chris Cheney
<XXChris.CheneyXX@xxxxxxxxx> wrote:


Note that, for example, the simple decimal fraction 0.1 is the recurring
binary fraction 0.0001100110011001100110011001100110 (0110 recurring) and
cannot therefore be stored precisely as a single/double/extended/real.

The FPU knows how to handle repeating fractions, regardless of their
base. The repeating portion is truncated (and rounded, if necessary)
at the last available digit of precision for any given float type. In
the case of "extended", the decimal portion terminates at the 18th
position.

I'm not sure what leads you to believe that 0.1 cannot be stored
precisely. Did you try it with code? I did...and it works fine.
I can store 0.1, and I get back 0.1
If you get back what you store, then by all accounts, it's precise.
.

Relevant Pages

  • Re: Is there a bigger data type in c than the double one?
    ... >> working with long doubles? ... As every binary fraction has a ... >> finite decimal fraction representation, ... > great precision because of the extended precision module, ...
    (comp.lang.c)
  • Re: Can someone please explain - real numbers
    ... recurring binary fraction 0.0001100110011001100110011001100110 (0110 ... at the last available digit of precision for any given float type. ... the decimal portion terminates at the 18th ... If you get back what you store, then by all accounts, it's precise. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Can someone please explain - real numbers
    ... binary fraction 0.0001100110011001100110011001100110 and ... at the last available digit of precision for any given float type. ... the decimal portion terminates at the 18th ... If you get back what you store, then by all accounts, it's precise. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Can someone please explain - real numbers
    ... binary fraction 0.0001100110011001100110011001100110 and ... The FPU knows how to handle repeating fractions, ... at the last available digit of precision for any given float type. ... inaccurate due to rounding errors. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Sql Everywhere Date precision ?
    ... As I said, i can simply store the "long" value, the result will be the same. ... I understand your need for precision, but as a practical matter I'm not ... The date's ticks is: Ticks 632902822536718750 long ... know if there is any way to "raise" the precision of the datetime format. ...
    (microsoft.public.sqlserver.ce)