Re: Why is the memory doubled when writing data to files?
Date: 27 Jan 2005 23:26:10 -0800
Shi Jin wrote:
> Hi there,
> I observed something that I cannot explain.
> Is there any explanation why the memory would get doubled when
> to files?
Yes. In the worst case, the size of a sequential unoformatted record
cannot be determined until the entire output list has been evaluated.
Most modern implementations of sequential unformatted I/O encode
records using a header that indicates the size of the record, followed
by the data, followed by a trailer that indicates the size of the
When writing to a file that does not allow seeks, the header must be
written before the data is written. Therefore, in the worst case, all
of the data must be buffered before the record can be written.
Consider the WRITE statement
WRITE (10) (F(I), I = 1, N), (G(I), I= 1, M)
where the external unit 10 is connected to a file that does not
allow seeks, and F is a function that modifies M. The processor
must evaluate the N calls of the function F before it knows the
size of the output record. The values returned by the calls
must be saved so that they can be written after the header.
The worst case almost never occurs in practice, but a complete
implementation must allow for the possibility. Many Fortran processors
will special case the cases where the size of the output record can be
determined in advance. Others special case output to files that allow
seeks. Either optimization eliminates the need to copy the data.
The simple fact is that a modern Fortran implementation is big. An
unbounded number of optimizations are possible. Vendors choose the
optimizations they implement based on their best guesses of users'
wishes. You should let the vendor know you would like his
to use less space for sequential unformatted output.