Re: storing an integer in a double precision

In article <ipv2j5$g2u$1@xxxxxxxxxxxxx>,
Lynn McGuire <lmc@xxxxxxxxxx> wrote:

Just a lack of understanding of conversion between integers and
double precision on my part. Back when we were primarily single
precision, we had to be very careful, using "ii = xx + 0.1" to
ensure that we did not loose the whole number value of a
incomplete conversion of 12,000 which was stored in the float
as 11.9999.

I have noticed lately that I have gotten lazy since we converted
the entire program to double precision and don't add the 0.1
conversion factor anymore as we don't seem to need it anymore.

Floating point numbers can always round down just as well as up. So
if you don't have exact integer values stored, but you want to
extract the nearest integer value, the intrinsic NINT() is probably
what you want. Also NINT() is supposed to work in the borderline
cases where adding a fudge factor before truncation fails due to
roundoff. NINT() has been in the language since f77.

$.02 -Ron Shepard
.

Relevant Pages

  • Re: IBM2435I on ROUND(x,-3)
    ... fraction. ... If an intermediate result is necessary, as in evaluation of an operational expression, the attributes of the intermediate result are the same as if a decimal fixed-point value of precision had appeared in place of the string. ... In other words, FIXED DECIMALwould be converted to FLOAT DECIMAL, and in the actual conversion process, any fractional part of the constant expressed in character form would influence the exponent of the converted result and the fractional digits would not actually be lost. ... ROUNDreturns the value of the real expression x rounded at a digit specified by n. ...
    (comp.lang.pl1)
  • Re: IBM2435I on ROUND(x,-3)
    ... fraction. ... If an intermediate result is necessary, as in evaluation of an operational expression, the attributes of the intermediate result are the same as if a decimal fixed-point value of precision had appeared in place of the string. ... In other words, FIXED DECIMALwould be converted to FLOAT DECIMAL, and in the actual conversion process, any fractional part of the constant expressed in character form would influence the exponent of the converted result and the fractional digits would not actually be lost. ... ROUNDreturns the value of the real expression x rounded at a digit specified by n. ...
    (comp.lang.pl1)
  • Re: double to int conversion yields strange results
    ... giving me two different values for the int. ... the question raised is not about the precision of the ... is done in extended precision. ... It is the conversion to 'double' before the conversion to 'int' that ...
    (comp.lang.c)
  • Re: printf and style g
    ... a G conversion specifier), ... Let P equal the precision if nonzero, ... if a conversion with style E would have an exponent ... formatting and a precision of 2. ...
    (comp.std.c)
  • Re: f77 vs f90
    ... On 11/5/12 4:06 PM, glen herrmannsfeldt wrote: ... processor to magically add precision to what is now always a single ... the standard doesn't exactly specify the conversion process from single to double. ... it does specifically define words that have a technical meaning that is either ambiguous or different from the normal every-day meaning. ...
    (comp.lang.fortran)