Re: Windows array allocation problem

Roy Lewallen wrote:
I'm getting an "insufficient virtual memory" error when trying to allocate an array to a size which is about 385 MB less than the amount of available virtual memory. I'm using the Intel Visual Fortran v. 9.1 compiler, with the compiler and program running under Windows XP.

The allocation statement is

ALLOCATE(A(N), STAT = IStat)

where A is a KIND = 4 complex array. The operation fails, with STAT = 41 (insufficient virtual memory) with N somewhere between 214,358,000 and 214,358,881. These correspond to allocated array sizes of 1,714,864,000 and 1,714,871,048 bytes respectively. (SIZEOF(A) shows the correct size after allocation even when the allocation returns STAT = 41.) The former succeeds, the latter fails. In both cases a call to Windows API function GlobalMemoryStatus or GlobalMemoryStatusEx immediately before the ALLOCATE statement returns a value of available virtual memory of 2,118,221,824 bytes. So I'm getting an "insufficient virtual memory" failure when trying to allocate the array to a size which is 384.665 MB less than the amount of reported available virtual memory. The operation succeeds when the allocated array size is 384.672 MB less than the amount of reported virtual memory, or less.

My questions are:

1. Why can't I allocate an array to a size much closer to the amount of available virtual memory? I realize, of course, that I have to allow for other memory use by the program in deciding the maximum I can safely allocate, but that's not the issue here.
2. If I won't be able to allocate nearly all the available virtual memory, how can I determine in advance how much I *can* safely allocate?

The Intel documentation says in numerous places that virtual memory problems should be addressed by adjusting the virtual memory setting in the Windows Control Panel. I'm a bit confused by this, because I'm under the impression that every 32 bit Windows program has an allocation of 2 GB of virtual memory. Isn't this the pool of memory the array is being allocated within? At any rate, changes to the Control Panel page file setting have no apparent effect on the problem.

Any insight would be greatly appreciated!

What are the consequences if you can't allocate 214,358,881 elements? Does it limit the size of the problem you can solve, or does affect your program's speed or accuracy? If the issue is speed or accuracy, how much of a difference does it make? How much would you lose if you just allocated, say, no more than 1 GB?

Louis
.

Relevant Pages

  • Re: Mergesort Vs Quicksort
    ... given as variable-length records packed end-to-end in one array, ... If your dataset is large enough that you can't allocate two full ... oodles of extra virtual memory you can allocate virtually for free, ... also allocating each CONS cell on the heap, ...
    (comp.programming)
  • Windows array allocation problem
    ... I'm getting an "insufficient virtual memory" error when trying to allocate an array to a size which is about 385 MB less than the amount of available virtual memory. ...
    (comp.lang.fortran)
  • Re: Date representation
    ... allocate the arrays, rewind the file and read in the data. ... the amount of data is not very large. ... Previously I just allocated some ridicously big array and it also ... Below you will find a routine to convert the dates to JULDAG. ...
    (comp.lang.fortran)
  • Re: Windows array allocation problem
    ... allocate an array to a size which is about 385 MB less than the amount of available virtual memory. ... In both cases a call to Windows API function GlobalMemoryStatus or GlobalMemoryStatusEx immediately before the ALLOCATE statement returns a value of available virtual memory of 2,118,221,824 bytes. ... So I'm getting an "insufficient virtual memory" failure when trying to allocate the array to a size which is 384.665 MB less than the amount of reported available virtual memory. ...
    (comp.lang.fortran)
  • Re: memory leak
    ... MSFT have already said that they will fix this problem, ... What is your definition of "video RAM"? ... You could try to allocate and have it throw if you're out of virtual memory. ...
    (microsoft.public.dotnet.framework.compactframework)