Re: Memory Management

On 4/27/2007 10:18 PM, Shiraz wrote:

I need to keep a hash of arrays in memory and keep adding things to it and
when the hash has so many things in it that it can cause a perl out of memory
error, i want to prune it. almost like a cache.

So whatever gets flushed from the cache can be recalculated if necessary? You
haven't said how you're filling it, but take a look at the Memoize and
Memoize::Expire modules on CPAN. They *might* do what you want.

If not, my advice is to create a class for your cache with get/set methods. When
it gets too big, start undef'ing entries. Devel::Size will give you accurate
results but be slow. If your array refs are of roughly equal size, you could use
scalar keys on the hash for a faster (but crude) estimate.

-mjc


.

Relevant Pages

  • Re: C++ hash map container
    ... For hashing, the initial hash requires looking at the entire key being ... four levels are likely to be in the cache. ... memory will probably be slower than traversing whatever part of the ... tree fits in teh cache. ...
    (comp.lang.cpp)
  • Re: fast dictionary search algorithm
    ... > performance by the hash implementation itself. ... and can you please quote me back Moore's Law? ... >> external memory you are stuck behind a much slower curve. ... the L3 cache will be a lot slower ...
    (comp.programming)
  • Re: More optimisation
    ... If I want to perform actions on several large arrays, ... The best way is to look at the memory hiearchy of the processor you are using. ... Neither method uses caching particularly well. ... The organization of the cache system can still ...
    (comp.lang.c)
  • Re: best collection to use for fast lookups
    ... > How would this approach compare performance with a hash table or some ... Real containers do take up a little more memory than arrays, ...
    (comp.lang.java.programmer)
  • Re: automatic arrays versus saved, allocated arrays
    ... arrays, we get between 50% and 20% better overall performance ... when changing this idiom in a portion of the code responsible for ... I'd wager that the performance improvement is due to cache ... coherency and/or memory management issues. ...
    (comp.lang.fortran)