Re: [C] Program has a math bug

From: Gary Schenk (gwschenk_at_fuzz.socal.rr.com)
Date: 02/04/05 

Date: Fri, 04 Feb 2005 05:20:54 GMT

Jack Klein <jackklein@spamcop.net> wrote:
<snip>
>
> The answer already posted by Edd is absolutely correct, but I wanted
> to provide a little more detail on whay this happens the way it does.
>
> The prototype for abs(), in <stdlib.h> is:
>
> int abs(int j);
>
> So when the compiler processes your statement:
>
>> b = abs( zed.upper_left.y - zed.lower_right.y );
>
> First it performs the subtraction of one double value from the other.
> Since both values are doubles, it does not have to do any conversions
> before the subtraction, and the result is a double.
>
> Then, because the prototype for abs() informs the compiler that it
> takes one parameter of type int, it must convert the result of the
> subtraction to int to pass to the function. This conversion is well
> defined in C, as long as the value of the double is not too large to
> fit within the values that an int can hold. The fractional part of
> the double is just chopped off. There is no rounding at all.
>
> 3.0 converts to the int value 3.
> 3.1 converts to the int value 3.
> 3.9 converts to the int value 3.
> 3.99999 converts to the int value 3.
>
> So, as Edd suggested, include <math.h> and replace abs() with fabs(),
> which has the prototype:
>
> double fabs(double x);
>

Edd, thanks for pointing that out. Do I feel stupid. I should use my
copy of Harbison and Steele for more than collecting dust! That
explains why my original version worked perfectly, it was all
integers, and why it busted after switching it to doubles.

Jack, thanks for that analysis. very informative. 

-- 
Gary Schenk
remove 'fuzz' to reply

Relevant Pages