Re: Rounding error when converting from double to int

On Aug 4, 3:17 pm, mark <nos...@xxxxxxxxxx> wrote:
Steve McConnel, in his book "Code Complete", suggests converting money
amounts from double to integer in order to avoid rounding errors
during calculations.

I assume he meant a *LONG* integer. I declined a free copy of this
book, so I don't have the actual text of it available to me.

[...] So we made a money class where the money amount
is stored as an integer instead of a double.  The cents are stored in
the 10's and 1's position of the integer.

For example:  $1.23 is stored as the integer 123.  The cents portion
is 23, and the rest is the dollar portion.

However we are having trouble initializing this integer amount when
the original double amount contains fractions of a cent (due to some
interest calculations).  We want to round any fractional cents up to
the nearest cent.  We use a rounding scheme where values greater than
or equal to 5 are rounded up.

Below is our initialization function.  Note that we use the type
__int64 so we can hold larger numbers but the same problems happen
with type int also.    myCurrencyx100 is of the type __int64.

The code multiplies the double by 100 first, then it rounds to the
nearest 1, then it converts to int.
So if value = 9.495 we want myCurrencyx100 to be initialized to 950.
But the code sets it to 949.

void GCSMoney::init( const double& value )
{
    if( value < 0 )
        myCurrencyx100 = static_cast<__int64>(100.0 * value - 0..5);
    else
        myCurrencyx100 = static_cast<__int64>(100.0 * value + 0..5);

}

I experimented with the order of the calculations.  I changed it to
round to the nearest hundredth (cent), then multiply by 100, then
convert to int.  This will correctly set myCurrencyx100 to 950 when
the double value is 9.495.  But when I tried a double value of  1.115,
it rounded it to 1.11 when it should have rounded it to 1.12.  However
the first init method worked for 1.115.

The problem is that the compiler makes 1.115 into 1.1149999...
internally, so that it does indeed get correctly rounded. It just
doesn't match your "visual expectation". The way that the C language
typically interacts with IEEE-754 numbers you probably want the
following:

if (value < 0)
myCurrencyx100 = (__int64) (100.0 * INCR_VAR_BY_ULP(value,1) +
0.5);
else
myCurrencyx100 = (__int64) (100.0 * INCR_VAR_BY_ULP(value,1) -
0.5);

You can practically implement INCR_ONE_ULP as:

double INCR_VAR_BY_ULP (double p, int inc) {
__int64 v = *((__int64 *)&(p)) + inc;
return *((double *)&v);
}

Its a bit gross, and doesn't work for INFs or NANs but its not grosser
than your use of __int64 in the first place. It also stops working as
you lose resolution when the values go higher then 1e53, but that
probably won't be an issue.

void GCSMoney::init( const double& value )
{
    if( value < 0 )
        myCurrencyx100 = static_cast<__int64>(100.0 * (value -
0.005));
    else
        myCurrencyx100 = static_cast<__int64>(100.0 * (value +
0.005));

}

This second one is closer, as you want to do the shift first, before
the scale, since that's the point at which you can correct IEEE-754
round-off effects. If you scale first, then you have a harder time
finding the ULP. For example, replace 0.005 above with 0.0050001 and
0.00499999 say, and you will see that it fixes pretty much all the
problems you will encounter. The ULP fix I show above is a little
more exacting with the representational anomolies but is clearly not
portable (neither is the use of __int64, BTW).

So I have two different methods, but neither works with all values.
For each of these methods, I noticed if I recoded the steps as
individual lines of code, the error happens during the step when
converting the double to int.

Is there any commonly accepted practice for handling these types of
errors?

You could use strings as your representational type instead of
doubles, and avoid the round off from doubles in the first place.

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/
.

Relevant Pages

  • Re: Rounding error when converting from double to int
    ...  So we made a money class where the money amount ... However we are having trouble initializing this integer amount when ... interest calculations). ...     else ...
    (comp.lang.c)
  • Rounding error when converting from double to int
    ... Steve McConnel, in his book "Code Complete", suggests converting money ... So we made a money class where the money amount ... interest calculations). ... then it converts to int. ...
    (comp.lang.c)
  • Re: looping
    ... If he does answer it will be some off topic rant aimed at distracting us away from the facts in the case in point also - it won't come anywhere near addressing the point I'm making that for running totals cursors rock and set based utterly sucks. ... (select sum(amount) ... from Trans t2 ... salesperson_id int not null, ...
    (microsoft.public.sqlserver.programming)
  • Re: looping
    ... id), id, salesperson_id, tran_date, clear_date, amount, ... from Trans t2 ... salesperson_id int not null, ... I've not done cursors in MS ...
    (microsoft.public.sqlserver.programming)
  • Re: looping
    ... id), id, salesperson_id, tran_date, clear_date, amount, ... from Trans t2 ... salesperson_id int not null, ... I've not done cursors in MS ...
    (microsoft.public.sqlserver.programming)