Re: Need modern version of old technique to show missing values



On 2009-01-07, Terence <tbwright@xxxxxxxxx> wrote:
I am rewriting the BMD program set (groans).

I have a master menu program that shows all the classes of programs by
pages and allows selection of any program out a class, to be executed
as a daughter program, returning to the menu program, and potentially
(well, the tests work) can call any selected BMD program and locate
and open appropriately, all the files each program will or might use
and open them as formatted or unformatted as required.

Personally, for interactive statistics work I really like R.

BUT: several programs specialise in statistics for incomplete data
("missing values").

The above-mentioned R has the symbol "NA" to indicate missing values,
distinguished from the usual IEEE754 floating point special values
such as Inf and NaN. Though I have to admit I don't know the low level
implementation of this.

Now: an ascii or formatted file record can be checked in each field
for "just blanks" to indicate the value is unknown.
So here I interpose a read of a character string and then process it
for "blank" fields before extracting values. But not so, in general
today, when reading binary files or unformatted file records.

Moreso today than during the 'old days', processors generally support
the IEEE754 floating point standard, which requires both positive and
negative zero.

In what I now have to call "the old days", you could have a positive
zero for a real value of zero and a negative real zero to indicate
this "missing value". These BMD programs utilized that trick.

As an aside, as of Fortran 95, the SIGN intrinsic is allowed to be
able to distinguish between positive and negative zero. But since you
seem determined to use an older standard this option is of course not
available to you, so it's rather academic.

Can someone suggest me, a way of coding a binary representation of a
floating-point field in such a way as I can detect that this value is
to indicate "not known", without confusing the arithmetic of the
statistics?
Obviously any user "today" will have to be able to easily produce his
data with this interior coding.
I need a coding method such that a simple test can decide if this
really is a floating point value or else the signal that the value is
"missing".

I suspect that inventing your own floating point format will not be
met with universal applause. Perhaps you can instead reuse one of the
IEEE754 special values to also mean missing? Or maybe keep track of
missing values in a separate data structure?

And I don't want the program to crash or have to trap conversion
errors via a program interrupt.

Since it's your project, you just have to program it such that missing
or invalid values don't cause a crash, then.

(i.e. I need F66/F77 methods so ai
have one set of Fortran source for any compiler and target computers;
one quarter to one half of which are AT/386/486 level)

?? Sounds like an awful lot of low level system dependent code.

One more question:
Is the matrix MAT1(20,30) equivalent to MAT2 (20,30,1) ?

The memory for the actual data, yes, but the descriptors the compiler
creates will be different. But the memory layout is essentially what
matters if you limit yourself to F77 and implicit interfaces (meaning
that when you pass an array argument, you are in practice passing the
address of the first element). Some compilers might be clever enough
to detect the interface mismatch and complain, though. Again, with a
newer standard you could use the RESHAPE intrinsic to do this in a
standard conforming way.


--
JB
.



Relevant Pages

  • Re: Rfd: floating point truncation V1
    ... Some other standard I looked at makes "round towards zero" ... an internal representation of floating point negative zero, "-0E", which differs ... to go for an IEEE FP extension). ...
    (comp.lang.forth)
  • Cfv: Floating point truncation
    ... RfD: Floating point truncation ... which, respectively, provide rounding to the nearest integer, and rounding to the nearest integer on the side of the number line towards negative infinity. ... ANS Forth does not provide a word for rounding towards zero, and returning a floating point representation of the resulting integer. ... Include a standard word, named "FTRUNC", to perform "truncation", i.e. rounding of a floating point number towards zero, returning a floating point representation of the resulting integer. ...
    (comp.lang.forth)
  • Re: Rfd: FNEGATE specification for signed zero
    ... The ANS Forth standard does not currently specify the behavior of FNEGATE for floating point zero on systems which support floating point signed zero. ...
    (comp.lang.forth)
  • Re: Rfd: FNEGATE specification for signed zero
    ... The ANS Forth standard does not currently specify the behavior of ... FNEGATE for floating point zero on systems which support floating ...
    (comp.lang.forth)
  • Re: Redux: Was there a negative real zero possibility in early Fortran?
    ... Was there a negative real zero possibility in early Fortran? ... Was this a "PDP thing" or an implementation device of esrly Fortran? ... Because the program, which mentions treatment of missing values, isn't ... The FORTRAN 66 standard said that the result of the SIGN ...
    (comp.lang.fortran)