Re: Floating point bug?

On Fri, 15 Feb 2008 17:30:03 -0800, Jeff Schwab wrote:

But good advice becomes a superstition when it becomes treated as a
law: "never test floats for equality". That's simply not true. For
example, floats are exact for whole numbers, up to the limits of
overflow.

That's not true. Epsilon becomes large (in absolute terms) for large
numbers. For 64-bit IEEE floats, eps > 1 at about 10**16.

Ah yes, good point. Sorry for my brain-fart, I was conflating integer
floats with the comp data type, as used in the Standard Apple Numerics
Environment (SANE). It used a floating point data type to store what was
effectively a 64-bit integer, which was quite significant in the days
when PCs used 16-bit integers!

Nevertheless, I stand by my claim, and support it by quoting from
Professor W Kahan of Berkley, who wrote in the forward to the Apple
Numerics Manual (2nd Edition):

[quote]
.... because so many computers in the 1960's and 1970's possessed so many
different arithmetic anomalies, computational lore has become encumbered
with a vast body of superstition purporting to cope with them. One such
superstitious rule is "*Never* ask whether floating-point numbers are
exactly equal."
[end quote]

In practice, you often do want to test floating point values within some
tolerance. But it isn't a universal law: see also "What Every Computer
Scientist Should Know About Floating Point Arithmetic".

[quote]
Incidentally, some people think that the solution to such anomalies is
never to compare floating-point numbers for equality, but instead to
consider them equal if they are within some error bound E. This is hardly
a cure-all because it raises as many questions as it answers.
[end quote]

http://docs.sun.com/source/806-3568/ncg_goldberg.html


Not directly related to this issue, but to get a great overview of some
of the problems with floating point, you could do a lot worse than to
read the following interview with Kahan:

http://www.ddj.com/184410314

and this wonderful lecture:

http://www.cs.berkeley.edu/~wkahan/ieee754status/ieee754.ps

where he details how optimizing compilers cause arithmetic errors, how
and why the facilities provided by IEEE arithmetic are underused, and
finally gives his idea of some things that could be done to turn the
situation around.

Sadly this lecture was given almost twelve years ago, and things have
barely changed. Compilers still do the wrong thing, especially optimizing
ones; computations that would be easy with NANs and infinities are
needlessly difficult; and benchmarks still over-value speed and under-
value getting the right answer, let alone simplicity of programming.


If you know your floats are whole numbers, and still writing
something like this:

x = 1234.0
y = 1000.0 + 200.0 + 30.0 + 4.0
if abs(x-y) < 1e-12:
print "x and y are equal"

then you are wasting your time and guilty of superstitious behaviour.

In what way? It's true that you hard-coded integers for which the
margin of error happens to be < 1,

No, in this case the error is *precisely* zero. There simply are no
rounding errors in this calculation, and the error is zero. If you wrote
the test as "if abs(x-y) < 1.0" you'd still be wasting your time.

It's true that I gave a hard-coded example, but it is hardly a special
case. There are many problem domains that don't require the full range of
floats and the type of rounding error you give can't occur.

(If your application calculates the amount of concrete needed to build a
house, say, then you pretty much know the amounts aren't going to be
measured in the trillions of megatonnes. If a user insists on specifying
that the house has 1.73e820 stories built on a base measuring 1.82e-87
metres squared, then roundoff is the least of your problems: the
algorithms you are applying will no longer be valid.)

Or possible you have already rounded the numbers yourself, earlier:

x = round(1.5678, 2)
x == 1.57
True

Why would you do this instead?

abs(x - 1.57) < 1e-12
True


The important thing is that your numbers have appropriately similar
scales. If you know that it going to be the case, then you know that
addition won't cause the sort of round-off error. I'm talking about.
Naturally there may be other forms of round-off, but rounding error
doesn't just appear from nowhere, it is predictable and understandable.


--
Steven
.

Relevant Pages

  • Re: Strange, very strange
    ... floating point numbers *do* have equality. ... Please stop spreading myths about floats about; ... understood by the general programmer already... ...
    (comp.lang.tcl)
  • Re: Annoying behaviour of the != operator
    ... Thats another pitfall, I'll grant, but its a pretty well known one to anyone with programming experience. ... I remember the old Standard Apple Numerics Environment" making the claim that testing equality on Macintoshes was safe. ... So it is quite possible to work with floats for *ages* before being bitten by this. ... Suppose your floating point numbers have six decimal digits of accuracy. ...
    (comp.lang.python)
  • Re: [TestUnit pack] Unequal equality
    ... Welcome to floating point math. ... Equality is a no-no with floats. ...
    (comp.lang.ruby)
  • Re: Floating-point arithmetic in CL
    ... > rationals, not as floating point. ... wants binary floating point, he should specify his constants in ... Programmers were *acutely* aware of the difference ... floats add at the same speed as integers. ...
    (comp.lang.lisp)
  • Re: Dan Bruce on guitar
    ...  In some respects McLaughlin floats in that he ... doesn't anchor with his pinky. ... at least over the bridge. ... Now, if that is the only benefit you derive from floating, ...
    (rec.music.makers.guitar.jazz)