Re: What is RoundToEx?

Dennis,

| You suggest we use this algorithm
|
| Result := Round(Input*100)/100;
|
| How does that compare to the one we use?
|
| Result := Round(AValue / LFactor) * LFactor;
|
| And what are the differences regarding precision?

If one had infinite precision it would not make any difference.

However, with fixed register(s) of any given length, the formula
presently used, in simple terms, can "lose" precision with the
parenthesized divide and that "error" is then "amplified" by the
post-multiply.

John's suggested formula "increases" [precision] accuracy with the
parenthesized multiply before the post-divide "loses" some of it.

--
Q

12/27/2007 10:31:42

XanaNews Version 1.17.5.7 [Q's Salutation mod]
.

Relevant Pages

  • Re: Crystal structure without diffraction?
    ... Compare that with a modern craftsman testing the standard ... undergad experiment. ... predict decay rates with worthwhile precision? ... lack of worthwhile measurements that can be made with better accuracy and ...
    (sci.physics)
  • Re: What is RoundToEx?
    ... You suggest we use this algorithm ... How does that compare to the one we use? ... And what are the differences regarding precision? ... Best regards ...
    (borland.public.delphi.language.basm)
  • Re: EGNOS broadcast schedule
    ... ESA is also broadcasting live signals on PRN124 (or 37 on ... I am not sure, by the way, whether precisions of different GPS receivers ... average claimed a better precision than my new 60cx but if I compare ...
    (sci.geo.satellite-nav)
  • Re: precise numeric type
    ... decimal point that supports addition and subtraction but does not ... have precision issues like floating point. ... and compare some _variant_t VT_DECIMAL values. ...
    (microsoft.public.vc.language)
  • Re: f**king mouse
    ... precision that enormously exceeds the precision with which you can ... But this is irrelevant to the physical world, since you cannot compare ... some abstraction instead of physical objects, ...
    (uk.misc)