Re: Rounding off double precision

In article
<3277fc93-726f-428f-82cd-aae971d0711f@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Bamm <bammster@xxxxxxxxx> wrote:

I'm trying some calculations in double precision, and I'd like the
output rounded off so that errors in calculation are not shown. For
instance, the result of COS(90 * 3.141592653589793238d0 / 180.) is
-4.37113886E-06. I'd like it to be zero. Thanks for any help.

The closest number is "never" going to be zero because there are
simply too many nonzero numbers near zero compared to floating point
numbers near the exact value of pi/2. However, I get
6.12323399573676604E-017 for the result of your expression, which is
a lot closer to zero than your value.

$.02 -Ron Shepard
.

Relevant Pages

  • Re: On writing negative zero - with or without sign
    ... neither Y/A or X/A underflows, and the sign bit wouldn't be set ... For nearly all calculations that's a lots bigger than zero. ... properly interpret the results of a calculation. ... Again, I disagree. ...
    (comp.lang.fortran)
  • Re: Product Calculation in an Access Query
    ... If "value" contains only positive numbers(no nulls, no negatives, no zero ... how do I do this calculation in Access? ... VLDT, Index Code, Index Name and Return. ... it doesn't appear that there is a product function in access. ...
    (microsoft.public.access.queries)
  • Re: Does DateDiff Have A Bug
    ... Now I just use the results from the query and everything works better than ... MemberID FirstName LastName EntryDate DaysRemaining ... calculated field within it's calculation) would look like; ... if there are zero days left which in ...
    (microsoft.public.access.formscoding)
  • Re: Does DateDiff Have A Bug
    ... Now you would go to Queries/Create query in Design View, ... MemberID FirstName LastName EntryDate DaysRemaining ... calculated field within it's calculation) would look like; ... I'm trying to write the result of lngDays when it reaches zero days ...
    (microsoft.public.access.formscoding)
  • Re: Does DateDiff Have A Bug
    ... Now you would go to Queries/Create query in Design View, ... MemberID FirstName LastName EntryDate DaysRemaining ... calculated field within it's calculation) would look like; ... I'm trying to write the result of lngDays when it reaches zero days ...
    (microsoft.public.access.formscoding)