Re: Rounding off double precision

Bamm wrote:
On Mar 29, 11:43 am, Bamm <bamms...@xxxxxxxxx> wrote:
I'm trying some calculations in double precision, and I'd like the
output rounded off so that errors in calculation are not shown. For
instance, the result of COS(90 * 3.141592653589793238d0 / 180.) is
-4.37113886E-06. I'd like it to be zero. Thanks for any help.

You may want to investigate using fixed-point arithmetic?

http://en.wikipedia.org/wiki/Fixed-point_arithmetic

For that matter, I also wonder why the result of Cosine is real and
not double precision. Thanks for any clarification.

Don't know; here's what I'm seeing:

program precision_trial
integer, parameter :: dp = selected_real_kind(15,307)
integer, parameter :: sp = selected_real_kind(6,37)
write(*,*) cos( 90 * 3.141592653589793238_dp / 180. )
write(*,*) cos( 90 * 3.141592653589793238_sp / 180. )
end program precision_trial

$ g95 precision_trial.f90 && ./a.out
6.123256244561421E-17
-4.371139E-8

$ gfortran precision_trial.f90 && ./a.out
6.123233995736766E-017
-4.3711388E-08

$ g95 -v
gcc version 4.0.3 (g95 0.90!) Mar 7 2008

$ gfortran --version
GNU Fortran (GCC) 4.3.0 20070713 (experimental)

Regards,
--
http://twitter.com/bil_kleb
.

Relevant Pages

  • Re: Rounding off double precision
    ... Bamm wrote: ... output rounded off so that errors in calculation are not shown. ... I'd like it to be zero. ... $.02 -Ron Shepard ...
    (comp.lang.fortran)
  • Re: On writing negative zero - with or without sign
    ... neither Y/A or X/A underflows, and the sign bit wouldn't be set ... For nearly all calculations that's a lots bigger than zero. ... properly interpret the results of a calculation. ... Again, I disagree. ...
    (comp.lang.fortran)
  • Re: Product Calculation in an Access Query
    ... If "value" contains only positive numbers(no nulls, no negatives, no zero ... how do I do this calculation in Access? ... VLDT, Index Code, Index Name and Return. ... it doesn't appear that there is a product function in access. ...
    (microsoft.public.access.queries)
  • Re: How do I replace error messages in Access?
    ... It would help if you posted the calculation you are using. ... Of course if you always wanted to return 1 when SomeNumber was zero, ... In my reports I often have to calculate the ... only if users update the field. ...
    (microsoft.public.access.queries)
  • Re: Rounding off double precision
    ... I'd like it to be zero. ... You can't get "output rounded so that errors in calculation are not ... appropriately if you use an F edit descriptor. ... unless the number is indeed exactly zero. ...
    (comp.lang.fortran)