Re: memory leak problem with arrays

sonjaa wrote:
I've created a cellular automata program in python with the numpy array
extensions. After each cycle/iteration the memory used to examine and
change the array as determined by the transition rules is never freed.

Are you aware that slicing shares memory? For example, say you defined
a grid to do the automata calculations on, like this:

grid = numpy.zeros([1000,1000])

And then, after running it, you took a tiny slice as a region of
interest, for example:

roi = grid[10:20,10:20]

Then deleted grid:

del grid

Then stored roi somewhere, for example:

run_results.append(roi)

If you do this, the memory for the original grid won't get freed.
Although grid was deleted, roi still contains a reference to the whole
1000x1000 array, even though it's only a tiny slice of it. Your poorly
worded description--no offense--of what you did suggests that this is a
possibility in your case. I recommend you try to create a new array
out of any slices you make, like this (but ONLY if the slice doesn't
depend on the memory being shared):

roi = numpy.array(grid[10:20,10:20])

This time, when you del grid, there is no object left referencing the
array data, so it'll be freed.

This might not be your problem. Details are important when asking
questions, and so far you've only given us enough to speculate with.

Carl Banks

.

Relevant Pages

  • Re: Virtual grid
    ... just ditch the array & load the data directly to the grid. ... Populating the grid from a file will be somewhat slower than from an array (memory), ... > want to display it in a flexgrid control. ...
    (microsoft.public.vb.controls)
  • Re: memory leak problem with arrays
    ... change the array as determined by the transition rules is never freed. ... Are you aware that slicing shares memory? ... a grid to do the automata calculations on, ... even though it's only a tiny slice of it. ...
    (comp.lang.python)
  • Re: union with packed struct
    ... >:You can slice it by the bit, you can slice it by GiG. ... > - when using character instructions, ... It had 128KB of addressable memory, ... array of 64K 16-bit words. ...
    (comp.lang.c)
  • Re: Question of memory management in Perl
    ... off the top of my head I would suggest you slice off the slice off the pieces ... that you have used already when pulling data from the original array. ... > I have question with respect to memory utilization using perl. ... The problem I get is in the middle of subroutine ...
    (perl.beginners)
  • Re: Fast string operations
    ... Looping: I thought looping over arrays in managed code was "slow" ... array handling and such. ... The problem with TrimHelper is that it always returns a new string instance. ... The customer perceives this as a memory leak. ...
    (microsoft.public.dotnet.languages.csharp)