Re: Deviation of double

From: Victor Bazarov (v.Abazarov_at_comAcast.net)
Date: 05/17/04 

Date: Mon, 17 May 2004 10:32:37 -0400

Christian Meier wrote:
> "Karl Heinz Buchegger" <kbuchegg@gascad.at> schrieb im Newsbeitrag
> news:40A8BEC4.A72FD084@gascad.at...
>
>>Christian Meier wrote:
>>
>>>Hallo NG
>>>
>>>My problem is the deviation of the floating point datatypes. 90.625 is
>
> not
>
>>>exactly 90.625. It is 90.624999999.... on my system. Now I tried to find
>
> a
>
>>>solution for my problem but I couldn't think of one.
>>
>>Forgive me. What was your problem?
>>
>>
>>>So I tried to write a
>>>work-arround which should solve the problem for the most numbers.
>>>What do you think of this: (I know it is not a perfect solution....)
>>
>>I doesn't look like solution to anything.
>>
>>--
>>Karl Heinz Buchegger
>>kbuchegg@gascad.at
>
>
> Sorry, wasn't a really good description of my problem. I will give more
> infos: I have a function with two parameters. One of type double, the other
> one is an int.
> string func (double dValue, int iDigits);
> This function must return a string which represents the double value.
> iDigits is the number of digits after the point. For this I use sprintf. And
> now I got a problem. When passing 90.625 to the function (iDigits = 2) then
> the function returned 90.62 instead of 90.63. I found out that the double is
> stored as 90.624999999... on my system. So I need a solution for this. The
> function has to return 90.63. So I thought I can add a small value to dValue
> before rounding. And the double has a limit of 15 decimal digits on my
> system. So I tried to write a work-arround as described in my first post.
> Can you now understand my problem?

I believe you're confused about rounding. 90.6249999999999 simply _must_
round to 90.62 if only two decimal digits are required. That's how
rounding is defined. You cannot assume that "since it's so close to
90.625, I should begin my rounding from 90.625 (instead of the actual value)".

If your function "has to return 90.63", add this statement:

     return 90.63;

to your function. If, OTOH, you want it to work correctly, analyse the
problem further. Why do you get 90.6249999999999 instead of [expected]
90.625? Why are you expecting 90.625? What values do you start with?
What operations are you performing on those values? And so on...

Victor

Relevant Pages

  • Re: This calculation is just wrong / computer cant count!
    ... BCD also has the advantage of providing more significant decimal ... arithemtic, AS IMPLEMENTED ON THE PENTIUM FPU, as demonstrated by my Floating Point ... If you can't understand that using binary to represent decimal digits ... properly rounding. ...
    (microsoft.public.vc.mfc)
  • Re: Rounding error when doing float-to-integer
    ... Agree with Glenn here. ... significant figures" step. ... The rounding is to 24 binary digits. ... The "7 decimal digits" bit is only a rough approximation that allows ...
    (comp.lang.fortran)
  • Re: Arithmetic operation bug
    ... Manoj Chourasia wrote: ... decimal digits are shown, taking rounding into account. ... doesn't round it), ...
    (comp.lang.ruby)
  • Re: Pixel Mania
    ... infinite number of decimal digits following the decimal point. ... truncated number that VB rounds to 15 displayed digits is an integer power ... Rounding in situations one doesn't expect. ... Most people would expect it to round first IMO. ...
    (microsoft.public.vb.general.discussion)