Re: Rounding off double precision

In article
<3277fc93-726f-428f-82cd-aae971d0711f@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Bamm <bammster@xxxxxxxxx> wrote:

I'm trying some calculations in double precision, and I'd like the
output rounded off so that errors in calculation are not shown. For
instance, the result of COS(90 * 3.141592653589793238d0 / 180.) is
-4.37113886E-06. I'd like it to be zero. Thanks for any help.

The closest number is "never" going to be zero because there are
simply too many nonzero numbers near zero compared to floating point
numbers near the exact value of pi/2. However, I get
6.12323399573676604E-017 for the result of your expression, which is
a lot closer to zero than your value.

$.02 -Ron Shepard
.

Relevant Pages

  • Re: On writing negative zero - with or without sign
    ... neither Y/A or X/A underflows, and the sign bit wouldn't be set ... For nearly all calculations that's a lots bigger than zero. ... properly interpret the results of a calculation. ... Again, I disagree. ...
    (comp.lang.fortran)
  • Re: Product Calculation in an Access Query
    ... If "value" contains only positive numbers(no nulls, no negatives, no zero ... how do I do this calculation in Access? ... VLDT, Index Code, Index Name and Return. ... it doesn't appear that there is a product function in access. ...
    (microsoft.public.access.queries)
  • Re: How do I replace error messages in Access?
    ... It would help if you posted the calculation you are using. ... Of course if you always wanted to return 1 when SomeNumber was zero, ... In my reports I often have to calculate the ... only if users update the field. ...
    (microsoft.public.access.queries)
  • Re: Rounding off double precision
    ... I'd like it to be zero. ... You can't get "output rounded so that errors in calculation are not ... appropriately if you use an F edit descriptor. ... unless the number is indeed exactly zero. ...
    (comp.lang.fortran)
  • Re: Chances of (random(0,n) + random(0,n) <= m)
    ... terms will be zero in the case m>n. ... > This is a death spell calculation that may be used in a game. ... > would be nice to be able to display the chance of killing the target. ... The numerator in Pis thus a triangular number ...
    (sci.math)