Re: opening multiple files in fortran

On Feb 18, 10:28 pm, Bob Walton <see....@xxxxxxxxxxxxxxxx> wrote:
------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You'll need to be a lot more specific than this if anyone is to help.
What *exactly* can't you seem to get to work? The internal write? Do
you get any error messages? If so, what *exactly* do they say? If not,
what exactly does the program do that you do not expect it to do, or
what exactly does it not do that you do expect it to do?

Yes, sorry - trying not to post a 30-page explanation, and that just
led to confusion.... The issue I am having trouble with is internal
write. I get an error right now, which is:

startint: error in format
apparent state: internal I/O
last format: r_hifreq10000.out
lately writing sequential formatted internal IO
Aborted

The file 'r_hifreq10000.out' is the first string in the file
'readin.dat' - readin.dat is a list of all the names of the input
files, so my intention was to take the input string from each line of
readin.dat, transfer that string over to a temporary filename, then
open this file, reading in all of that file's values to my large
array, outnu(1:14391,1:10000). Then, once I have the values in local
memory (in the array) I wanted to create 10,000 output files to write
the different elements of the array (for each of the 10,000 variables
- so each of my 10,000 output files would have 14,391 entries apiece).

This is basically just transforming a matrix transpose (I'm calling my
code transpose.f). My problem is with internal write, not with file-
open limitations; when I tried running these operations with perl and
python, both worked fine up until I hit this number of files. Oh, and
I'm running all of this on a linux machine with 2 GB of memory, and
the array elements will be double precision. Our machines can handle
that, but I need to be able to open and close the output files so that
I don't more than 14,000 files simultaneously open (that happens to be
our machines' hardwired limit, regardless of how much memory I'm
asking for).

And the complete code, with declarations (so far) of the variables,
and everything else looks like:

program transpose

implicit none
integer i,j,k
integer out_unit,num_files
integer fileindex,nuindex
parameter (out_unit=20,num_files=14391)
character (len=20) file_name
character (len=20) inname
character (len=20) outfile
real*8 innu,inrp
real*8 outnu(1:14391,1:10000)

fileindex=1
nuindex=1
open(12,file='readin.dat')
90 continue
read(12,*,end=99)inname
write(file_name,inname)
open(30,file=file_name,form='formatted')
100 continue
read(30,*,end=110)innu,inrp
outnu(nuindex,fileindex)=inrp
nuindex=nuindex+1
goto 100
110 continue
do i=1,14391
write(outfile,'(''nu'',i0,''.dat'')') i
open(15,file=outfile,form='formatted')
do j=1,10000
write(15,*)nuindex,outnu(i,j)
enddo
enddo
nuindex=1
fileindex=fileindex+1
close(15)

99 continue
close(30)


stop
end



2. nuindex is incremented, but never set back to 1 until the end. If
you are truly reading a large number of files, you will need to include
a means of resetting nuindex for each file.
3. Only one file is read. This conflicts with your statement that you
intend to read a large number of files.
4. fileindex is incremented only once. If you intended to loop back to
the start, you neglected to include that code.

I was intending the goto statements to serve as a loop here - read
in one input file (100 on) and fill the array, then once that's
finished reading in, go to 110 where it would write out those values
to 10,000 different files. That's within my system's limitations, but
it's true that I probably would be better off going back to 90 from
that point (instead of 110) and continuing to fill my array, then once
the array is completely filled, write out each individual file,
sequentially opening and closing.

I'm sure there are problems with my setup at this point, but the
thing which I am most worried about is the format/best way to use
internal write.
Thanks a lot for the help,
-a



5. After one file is read to end-of-file, the code proceeds to output.
This conflicts with your description of reading all the files first,
then outputting all the files.

what the best format for this is.... Any suggestions? Code looks
like:

fileindex=1
nuindex=1
open(12,file='readin.dat',form='formatted')
90 continue
read(12,*,end=99)inname
write(file_name,inname)

The above statement will treat 'inname' as a format string, and has no
I/O list. It doesn't look that is your intention (but, then again, I'm
not privy to the contents of file 'readin.dat')??

open(30,file=file_name,form='formatted')
100 continue
read(30,*,end=110)innu,inrpwr
outnu(nuindex,fileindex)=inrpwr
nuindex=nuindex+1
goto 100
110 continue

Here you have read one file, not 10000. But you're proceeding to output
anyway? Note that all entries in 'outnu' with the second subscript
greater than 1 are undefined.

do i=1,14391
write(outfile,'(''nu'',i0,''.dat'')') i
open(15,file=outfile,form='formatted')
do j=1,10000
write(15,*)nuindex,outnu(i,j)

The variable 'nuindex' here has the number of items read from the first
file at this point. Is that really what you want to output here?

enddo
enddo
nuindex=1
fileindex=fileindex+1
close(15)

Thanks!
-a

HTH.
--
Bob Walton
Email:http://bwalton.com/cgi-bin/emailbob.pl


.

Relevant Pages

  • Re: reading nth columns
    ... e.g suppose format of my file is something like ... I know of fscanf which can be used like ... succeeded in reading anything. ... into a chararray and using sscanfon the array; ...
    (comp.lang.c)
  • Re: reading nth columns
    ... e.g suppose format of my file is something like ... I know of fscanf which can be used like ... succeeded in reading anything. ... into a chararray and using sscanfon the array; ...
    (comp.lang.c)
  • Re: unbuffered file IO?
    ... and where im reading contents into the array ... since the array is a longand therefore strings can't be placed in it ... looking at files which have the special sort of format you're ...
    (microsoft.public.dotnet.languages.csharp)
  • 16 bit wav file reading
    ... I am reading a 16 bit wav file with format: ... i am looking to read the file into an array to manipulate it. ...
    (comp.lang.java)
  • Re: A Question about arrays file merging
    ... > I do have some programing experience-mostly working ... and in both cases load the output into an array? ... format you can use either language's I/O facilities [e.g. 'fopen', ... then I understand the portability concern. ...
    (alt.comp.lang.learn.c-cpp)