Re: Converting Long to Float: Values not Matching



Z wrote:
I am getting:

3250051236
3.25005133E9

I tried with multiple numbers but still the values are not equal. What
am I missing?

Thanks,

Floats have limited precision. Your number is larger than the number of the precision.

Internally, numbers are represented as a number (the mantissa) multiplied by some power of 2; your number cannot be exactly represented to the precision that the mantissa has, so it lops off a few bits at the end, giving you a moderately close but not exact approximation.
--
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth
.



Relevant Pages

  • Re: Why fixed point ?
    ... Yes, we do not care about lower half of result bits (standard precision result is in upper half), but in multiple accumulations we may loose carry bitfrom lower half. ... Most DSPs provide a MAC engine with an accumulator that has significantly more bits than a multiply. ... The correct technique to use this precision was to do your MAC, shift the result around as necessary to line up with the notation that you were using, then saturate the result based on the contents of the "overflow" part of the accumulator. ...
    (comp.dsp)
  • Re: calculation accuracy
    ... about inaccuracies in trig functions introduced by argument reduction. ... That is, sinfor large X, even when X should be close to a multiple ... The argument here is double precision, ... Then subtract. ...
    (comp.lang.fortran)
  • Re: Infinity + Infinity (or NegInfinity - NegInfinity)
    ... Extended precision is the easy answer, but if that isn't available (and ... all you need is some decent support for multiple ... above design helps a lot with optimising that); ...
    (comp.arch)
  • Re: In need of a closed form for this constant!
    ... multiple precison log vs multiply and divide???? ... All the algorithms I know of require several multiply operations, ... So: slower but not ... number of multiplies to achieve n-bit precision, ...
    (sci.math)
  • Re: In need of a closed form for this constant!
    ... multiple precison log vs multiply and divide???? ... All the algorithms I know of require several multiply operations, ... much slower. ... number of multiplies to achieve n-bit precision, ...
    (sci.math)