Re: Fortran memory allocation (stack/heap) issues

From: Andy Nelson (andy_at_kant.maths.ed.ac.uk)
Date: 04/29/04 

Date: Thu, 29 Apr 2004 10:34:49 +0000 (UTC)

Hi folks,

Thanks for the helpful responses. They meshed with and
cleared up some of the muddy thoughts/impressions I had
before. Here are some responses to them (both mixed together)
and some additional information.

>> -Under what circumstances would one expect that the call
>> might trigger a quantity to have a copyin/copyout sort of
>> arrangement and what might trigger a call by reference sort
>> of arrangement?

RM> ...

RM>Basically, you rarely see array copying in f77. (It can happen,
RM>but it is rare). You also don't tend to see it in what I would
RM>call "pure" f90 style. You do get it from some attempts to
RM>mix f77 and f90 styles.

Thanks for that. I wonder in passing if it would make sense for
vendors to create a switch that forced one or the other behavior.

I'm curious...since we are trying to migrate slowly to f90 type
of coding, and may end up with a what really amounts to
a mixed f77/90 style for a fair bit of code...what would you
define as a mixed style in terms of pitfalls for the unwary
to watch out for?

>> -There are also occasions when a routine might be called with
>> one or the other npart=0 (see code below). As a completely

RM>Zero-sized arrays are allowed in f90, but not in f77. I bet the
RM>compilers in question were f77 ones.

GH>I believe you would be safer to make it 1. As Richard said, 0 length
GH>arrays weren't legal in F77, so you can't be passing one that is

I vaguely remember a bit more about this, though it was a very quick
thing some time ago and I just wanted to make it work rather than
chasing its cause at the time. IIRC it was a Sun compiler (version?)
running in f77 emulation mode (i.e. using the f77 command for compilation,
but secretly or not-so-secretly actually using the f90 compiler
with some `pretend you are f77' options). In any case, it is very
useful to know that it will likely go away with the switch to an
f90-only sort of environment that we have planned.

As far as passing 1 vs 0, I thought at the time about doing exactly
this, but unfortunately, passing '1' isn't possible, due to code
specific issues. Basically, the code needs to be able to integrate
forward in time any number of particles of a given type, including
zero of one type and some of another. Therefore, as far as I can
figure out, there isn't really any way of triggering on when you
really mean to integrate one particle and when you secretly mean don't
integrate any because there are none in that simulation. Sigh.

>> Bear with me. The situation is that the code will not run due to
>> insufficient stack size, given a problem of some size that is
>> still much smaller than the total available memory.

GH>...It might just be that
GH>someone has set a small stack size.

My colleague reset the stacksize to as large as allowed, so
that was unfortunately not the case.

>> The machines where problems are found at the moment are a
>> Hitachi SR8000 and an Itanium2 machine put together by

GH>Interesting machines, and neither very popular. It is possible
GH>that you found a simple bug that just hasn't been noticed yet.

The code runs fine on these machines with smaller jobs, so it
is likely to be a `running out of space issue' like the one
both you and Richard point out:

GH>I would look carefully for local (not dummy arguments to the
GH>subroutine) arrays that might be causing stack overflow.

RM>One thing you might be running into has nothing to do with
RM>argument passing. Non-saved local arrays might plausibly be
RM>allocated on the stack. That includes "local" arrays in the main
RM>program. Many compilers have options to change that behavior.

There are several of these throughout the code. I've never
particularly counted them specifically though. Looks like its
time for an inventory, and to mark them SAVE as Glenn suggests.

>> they come from. There are no cases of allocate/free arrays in
>> any routines in the whole code. So I'm at least partly confused
>> as to why there might be issues of heap vs stack allocation to

GH>Both heap and stack are foreign concepts to F66 and F77, and
GH>you will even find discussion of them gets shot down in comp.lang.c.

Hence my confusion when the issue came up with the systems guy
who originally thought that stack/heap issues and how they relate
to argument passing were the source of the problem :-)

RM>P.S. When do you get the stack overflow? Imediately on loading
RM>the program or later? You should be able to get a hint from
RM>when it happens.

I asked my colleague, after seeing this question. I'm told
that it dies right at startup. That seems to indicate to me
that it didn't have much to do with array passing/copying into
subroutines that may only get called after some time later.
It also makes me wonder a little bit about whether the total
memory was sufficient, though from what I know of the job size
(footprint of 6-700MB or so, when I've tested it on other
machines) and the machines in question, that is less likely.

As you suggest, its time for some local array stock taking...

Thanks very much again for your (always useful) help.

Andy 

-- 
Andy Nelson                     School of Mathematics
andy@maths.ed.ac.uk             University of Edinburgh 
http://maths.ed.ac.uk/~andy     Edinburgh Scotland EH9 3JZ  U. K.

Relevant Pages

  • Re: the efficiency of STRUCTURE on SGI IRIX
    ... data on a IRIX machine with f77. ... since there are a large amount of groups ... here, about 10**6, and the number of sub groups diversifies between 0 ... arrays, I believe. ...
    (comp.lang.fortran)
  • Re: Fortran memory allocation (stack/heap) issues
    ... you rarely see array copying in f77. ... call "pure" f90 style. ... Zero-sized arrays are allowed in f90, ...
    (comp.lang.fortran)
  • Re: Matlab to modify a text file?
    ... > looking at f95 as well and I think your point on arrays might get me out ... Today, w/ upto GBs of memory, is there any ... The two texts I have on f77 and f90 both failed to mention the ... There are compiler-specific compiler ...
    (comp.soft-sys.matlab)
  • Re: Fortran pros and cons (was Re: Coding style article with interesting section on white space)
    ... > than in f77, but it is not obviously more expressive or less complex. ... here are some simple examples of the greater expressiveness of ... Fortran 95 compared to F77 or C for calculations involving arrays. ... The ufunc of Numeric Python and the map of basic Python offer similar ...
    (comp.lang.python)
  • Re: F77 & WIN XP
    ... system with a freeware F77. ... machine, the older F77 compilers will supply a default stack of 8k, ... the stack space will not be enough unless ou change ... however you want a Windows executable program as a result of ...
    (comp.lang.fortran)