Re: Question about precision

From: Paul Van Delst (paul.vandelst_at_noaa.gov)
Date: 08/25/04 

Date: Wed, 25 Aug 2004 16:31:55 -0400

Keith Refson - real email address in signature wrote:
> Paul Van Delst <paul.vandelst@noaa.gov> writes:
>
>
>>I have code where I need to interpolate several instrument response
>>functions to the same frequency grid and begin and end
>>frequencies. The data I read in from file is at a set spacing,
>>0.1cm^-1, but after computing that value from the begin and end
>>frequency and the number of points, the value is always something like
>>0.1000000000000002cm^-1. This has caused problems since the extra
>>0.0000000000000002 makes a test that I use to ensure the interpolation
>>is alway within the actual data (extrapolation is a no-no here),
>>fail.
>
>
> Might I suggest a different approach. Your problem arises because
> your grid spacing of 0.1 cm^-1 has no exact representation in binary
> floating point. But the numerical value depends on the units
> and can be changed to suit. If your program used internal units of
> 0.1cm^-1 then the grid spacing would be 1.0 which *is* exactly representable.
> In fact you could use integers which would eliminate representation
> and rounding errors entirely.

Hello,

Yeah, I've thought about that, but the frequency interval can be anything. 0.1cm^-1 is an
"easy" one. I could get data at a spacing of 0.135cm-1 or 0.0025cm-1 spacing. I don't
necessarily know what the value is in advance so I have to compute it (I compute the
average of all the intervals, rather than the difference of just two points). So to take
into account a variable frequency interval, I'd still have to do something similar to the
     dF_Exponent = INT( ABS( LOG10( Max_Delta_Frequency(l) ) ) ) + 2
     dF_Multiplier = TEN**dF_Exponent
     dF_Scaled = INT( Max_Delta_Frequency(l) * dF_Multiplier )
     Max_Delta_Frequency(l) = REAL( dF_Scaled, fp_Kind ) / dF_Multiplier
stuff I mentioned before so I can determine what a unit grid spacing actually means.

And, besides, the calculations themselves (interpolation and integration) are performed
correctly. The little left over bits I mentioned don't affect those (since they are so
small), they just affect the < and > tests I use to QC the inputs.

cheers,

paulv

Relevant Pages

  • Re: Resampling
    ... It's all interpolation. ... > doesn't matter where they are or what their spacing is. ... coefficient memory or sample computation than others. ... but it does make a difference on compute power what the SRC ...
    (comp.dsp)
  • Re: Indicator of the "difference" between two curves
    ... as differing only negligibly from exact calculation. ... spacing without sacrificing accuracy of interpolation. ...
    (sci.stat.math)
  • Question about precision
    ... I have code where I need to interpolate several instrument response functions to the same ... frequency grid and begin and end frequencies. ... interpolation is alway within the actual data, ... Printing out the values ...
    (comp.lang.fortran)