Re: Rounding off double precision

Bamm <bammster@xxxxxxxxx> wrote:

On Mar 29, 11:43 am, Bamm <bamms...@xxxxxxxxx> wrote:
I'm trying some calculations in double precision, and I'd like the
output rounded off so that errors in calculation are not shown. For
instance, the result of COS(90 * 3.141592653589793238d0 / 180.) is
-4.37113886E-06. I'd like it to be zero. Thanks for any help.

For that matter, I also wonder why the result of Cosine is real and
not double precision. Thanks for any clarification.

Well...

1. You can't get "output rounded so that errors in calculation are not
shown." That is equivalent to asking for there to be no errors in
calculations. Ain't gonna happen. If you can establish what the order of
magnitude of the errors will be, then you can round the output
appropriately if you use an F edit descriptor. You aren't ever going to
get exactly zero out of an E edit descriptor (or list-directed output)
unless the number is indeed exactly zero.

2. The result of that particular COS *IS* double precision. If you are
seeing something else, then either a) you are misinterpreting the cause
of what you are seeing, or b) it is a compiler bug. I'd place my money
on (a). Quite a lot of it, and at good odds. :-) You need to show
exactly what is leading you to this conclusion. That means more than
just an isolated expression and an alleged result. I'd need to see a
complete program, most definitely including all declarations and
whatever output statement you used to get this result. For example,
when I run the complete program

write (*,*) COS(90 * 3.141592653589793238d0 / 180.)
end

using g95 on this Mac, I get

6.123256244561421E-17

which is about the expected accuracy for double precision.

As an example of using an F edit descriptor to round, if I change the
program to

write (*,'(f15.12)') COS(90 * 3.141592653589793238d0 / 180.)
end

I then get

0.000000000000

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain
.

Relevant Pages

  • Re: On writing negative zero - with or without sign
    ... neither Y/A or X/A underflows, and the sign bit wouldn't be set ... For nearly all calculations that's a lots bigger than zero. ... properly interpret the results of a calculation. ... Again, I disagree. ...
    (comp.lang.fortran)
  • Re: Product Calculation in an Access Query
    ... If "value" contains only positive numbers(no nulls, no negatives, no zero ... how do I do this calculation in Access? ... VLDT, Index Code, Index Name and Return. ... it doesn't appear that there is a product function in access. ...
    (microsoft.public.access.queries)
  • Re: How do I replace error messages in Access?
    ... It would help if you posted the calculation you are using. ... Of course if you always wanted to return 1 when SomeNumber was zero, ... In my reports I often have to calculate the ... only if users update the field. ...
    (microsoft.public.access.queries)
  • Re: Chances of (random(0,n) + random(0,n) <= m)
    ... terms will be zero in the case m>n. ... > This is a death spell calculation that may be used in a game. ... > would be nice to be able to display the chance of killing the target. ... The numerator in Pis thus a triangular number ...
    (sci.math)
  • Re: Problem with zero result
    ... It seems unlike that you mean when the result of this expression is zero, ... could return anything other than Null if [Lunch Taken]-were ... it only *looks* like it is exactly zero. ... If the fields are not Date/Time fields, ...
    (microsoft.public.access.queries)