Re: Memory issues

Joost Diepenmaat a écrit :
jm <jm@xxxxxxxxx> writes:

Modifying a little bit again the script, and checking execution with
ltrace, I observed malloc is called 1871 times when free is just called
922 times.
Isn't it an issue?

Please keep in mind that perl's memory allocation strategy in general is
optimized for longer running programs, not for one-off scripts (which
makes sense, since one-off scripts don't usually need the performance
gains).

I did not read this, nor in the documentation, nor in the faq.
Might be a faqmemory might be helpful?

This means that for instance subroutines will get memory
allocated on the assumption that they'll be called again, and will take
about as much memory the next time.

But when a same routine is called several times, with different kind and
size of data, some times it consume lot of memory, and some other time
less memory.
If your code contains hundred of functions and twenty of them consume
200 Mbytes (in a similar way as aa in my example), then a 2 Gbytes
computer will not be enough to run it.

This is NOT a memory leak per se, but it does mean that if you have a
subroutine that takes 100Mb to complete, your program will take that
memory and probably not give it back until the program ends.

I understand this. But I'd like to have the opposite feature.
Or at least one perl function to release the memory used by a function
(or a package).

IOW, if you
have a long-running program that only means you need 100Mb for it to
run, it does NOT mean it takes a 100Mb for each call.

Yes, it is what I observed.

But this mean it is not possible to free the memory consumed by one
function, when you know you need memory in another one function.

In your test case, don't assume that just because the regular expression
replacements don't in theory *need* to use any additional RAM, they
won't. Especially not if you're using UTF-8 encoded strings (which you
are). Perl algorithms tend to exchange RAM for speed in most cases
anyway, and replacing a match with a new string of exactly the same
length in bytes in a unicode string is a pretty uncommon use-case, so
it's likely not optimized.

I do not think the issue is here.

Anyway, I've not seen a serious memory leak in perl itself in ages, and
I run perl processes that use up to 8 Gb of RAM and run for months
without issues.

This mean you have a 8 Gbytes RAM memory computer.

But if memory was used by perl in a better way, might be the same
programs might work on a 512 MBytes RAM computer.





.

Relevant Pages

  • Re: Steve Jobs demos Macintosh in 1984
    ... Woody wrote: ... Stop annoying the fucking hell out of me, ... For sure not much computer memory had been made in 1948 - but it could ... For sure you got a bit more RAM to use in the Speccy - ...
    (uk.comp.sys.mac)
  • Re: Windows uses 4 GB? Really?
    ... I bought 4 GB of RAM because I thought it would speed up my system a bit. ... Some info that is posted here as reference seems to suggest that it is a Windows issue, but others seem to suggest that it is also a hardware issue. ... "I have install total 4GB memory on my motherboard. ... On a SLI system, since PCI-Ex graphic cards will occupy around 256MB, another 256MB will be occupied after you install a 2nd PCI-Ex graphic card. ...
    (microsoft.public.windowsxp.general)
  • Re: Computer wont boot up after installing new RAM
    ... I even took a look at an old Gigabyte motherboard ... DDR2 memory stick, the notch does not line up. ... "To install the memory module, ... into the motherboard RAM slots. ...
    (microsoft.public.windowsxp.hardware)
  • Re: Random Restarts - Please Help
    ... I upgraded my BIOS a long time ago ... Beginners Guides: Diagnosing Bad Memory ... How to diagnose bad RAM ...
    (microsoft.public.windowsxp.help_and_support)
  • Re: Alocated Memory Error (SQLSERVR)
    ... but my understanding is that the MSDE instances are designed to ... and you don't want it grabbing all that RAM. ... running but non ever going about 200mb of memory usage. ... Server 2000 SP4, ...
    (microsoft.public.windows.server.sbs)