Re: Iteration and double

Christopher Smith <csmith@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
I am iterating through a loop, and iterating not by an integer but by a
decimal value, in this case 0.00025. However, after a number of
iterations, the value is off by 0.00000000000001. What's the best way to
round this back to the nearest 10-E5, or does it not matter???

Noting Patricia's response (which is exactly spot on, and you should
read it if you haven't), you can do something like this:

public double fix(double v)
{
if (Double.isNan(v)) return v;
double t = Math.round(v * 1.0e5);
if ((t == Long.MAX_VALUE) || (t == Long.MIN_VALUE) return v;

return t / 1.0e5;
}

This takes a value that may have some rounding error, and makes it as
close as possible to a decimal number with at most 5 fractional digits.
There are three things to remember, though:

1. This does NOT cause the value to become equal to a decimal number
with at most 5 fractional digits. In most cases, there is no value of
type double that is equal to such a number.

2. This is not the way to display a decimal number. For display, use
java.text.DecimalFormat to convert to a string with the desired
formatting.

3. If the magnitude of the double is large enough, then it may be that
the double is not sufficiently precise to hold even five fractional
digits. If this is the case, then the code above returns the value
unchanged.

There are really very few reasons to do this. The only one that comes
to mind is that you have done some calculations for which you are sure
(because of your knowledge of the problem) that the result is a nice,
round decimal number with relatively few decimal places, and you want to
correct any rounding error. However, in almost all cases where you need
to minimize rounding errors on nice round decimal numbers, it is
appropriate to use java.math.BigDecimal instead. I suppose there may be
a few intermediate cases where performance is important enough to
justify doing the complicated auditing of all operations to ensure that
the accuracy is good enough with floats, and then eliminate the error
with something like the above.

--
Chris Smith
.