Re: Why the nonsense number appears?

Johnny Lee wrote:
print time1, time2 

1130748744.461 1130748744.500

float(time2) - float(time1) 

0.039000034332275391


Why are there so many nonsense tails? thanks for your help.

http://en.wikipedia.org/wiki/Floating_point#Problems_with_floating-point, especially 'Rounding'. Or google for "gloating point precision" if you need more details.

Daniel
.

Relevant Pages

  • Re: demonic numbers !
    ... > corresponding calculation of error intervals. ... Then they're either not doing enough math for rounding errors to ... accumulate or they're not using a wide enough range of magnitudes. ... They don't need more than 53 bits of precision to store those values. ...
    (comp.lang.lisp)
  • Re: Why (0.09+0.01-0.1) is not equal to 0.09+0.01-0.1 ?
    ... The binary to decimal conversion gives full accuracy by default, but also includes rounding algorithms since users will seldom care about hundreds of figures. ... Function D2BAs String ... the work involved in avoiding lost precision due to type conversion ... ' D2Dis sufficient to uniquely determine the binary representation of x ...
    (microsoft.public.excel)
  • Re: The Okopochini Bug
    ... Knowledge base articles for full coverage of difficulties that Excel ... How to Correct Rounding Errors in Floating-Point Arithmetic ... As you have noticed Excel handles only 15 digits of precision. ... reason is that Excel, like many other computer programs, uses the IEEE ...
    (microsoft.public.excel.misc)
  • Re: Why (0.09+0.01-0.1) is not equal to 0.09+0.01-0.1 ?
    ... > gives full accuracy by default, but also includes rounding algorithms ... > partitioned mantissa by without risk of overflow ... > Function D2BAs String ... the work involved in avoiding lost precision due to type ...
    (microsoft.public.excel)
  • Re: Bad math when using * operator along with Math.pow
    ... java.math.BigDecimal will give you some more control over precision. ... "Nick Hauenstein" wrote in message ... > (to get around .NET's banker's rounding). ... Ignore any weird variable names (like sngPi when it's the double ...
    (microsoft.public.dotnet.vjsharp)