Re: Memory issues

On 2008-03-30 09:28, jm <jm@xxxxxxxxx> wrote:
Joost Diepenmaat a écrit :
jm <jm@xxxxxxxxx> writes:
Modifying a little bit again the script, and checking execution with
ltrace, I observed malloc is called 1871 times when free is just called
922 times.
Isn't it an issue?
[...]
IOW, if you
have a long-running program that only means you need 100Mb for it to
run, it does NOT mean it takes a 100Mb for each call.

Yes, it is what I observed.

But this mean it is not possible to free the memory consumed by one
function, when you know you need memory in another one function.

You can. Perl keeps around the lexical variables, but not any objects
they point to. So avoid large scalar, hash or array variables in subs
and use references instead. For example, compare the behaviour of

#!/usr/bin/perl
use warnings;
use strict;

print a(@ARGV), "\n";
exit 0;

sub a {
print "entering a @_\n";
my ($n) = @_;

my $s1 = "a" x $n;
my $s2 = "b" x $n;

my $rc = length($s1 . $s2);

print "leaving a @_\n";
return $rc;
}

and

#!/usr/bin/perl
use warnings;
use strict;

print a(@ARGV), "\n";
exit 0;

sub a {
print "entering a @_\n";
my ($n) = @_;

my $s;
$s->[1] = "a" x $n;
$s->[2] = "b" x $n;

my $rc = length($s->[1] . $s->[2]);

print "leaving a @_\n";
return $rc;
}

(And of course "length($s->[1] . $s->[2])" is (intentionally) stupid -
replace it with "length($s->[1]) + length($s->[2])")


Anyway, I've not seen a serious memory leak in perl itself in ages, and
I run perl processes that use up to 8 Gb of RAM and run for months
without issues.

This mean you have a 8 Gbytes RAM memory computer.

But if memory was used by perl in a better way, might be the same
programs might work on a 512 MBytes RAM computer.

Yes. But memory allocated to lexicals is usually the least of your
worries in this case. The overhead of typical perl data structures is
much worse. (Just this month I reduced the memory consumption of a
program from about 3 GB (which meant that it crashed sometimes, since
that's the limit on 32bit linux) to less than one GB by replacing an
anonymous array with a string (which I always had to unpack and repack
to access and manipulate the data within, which is ugly, but not really
slower than accessing the array).

hp
.

Relevant Pages

  • Re: Max size of an array used in perl.
    ... Oses these days aren't limited by RAM. ... Disk is slower than RAM so the machine will go to a crawl. ... And, unfortunately, there are some things that are easy to do in Perl ... structures in memory instead of reverting to using a database. ...
    (comp.lang.perl.misc)
  • Re: Perl garbage collector
    ... Does your OS have a function that allows processes to return memory to ... As you see some RAM is given back to the OS ... I would expect that the Perl virtual machine ... memory allocation and garbage collection. ...
    (perl.beginners)
  • Re: Exploiting the Perl
    ... CO> snip ... CO> data structures were leaked by the Perl program and continued to take ... CO> up memory, which would mean if you could get the address of the memory ... stored to disk and later reloaded in to ram and usable. ...
    (perl.beginners)
  • Re: Memory issues
    ... I observed malloc is called 1871 times when free is just called ... Please keep in mind that perl's memory allocation strategy in general is ... Or at least one perl function to release the memory used by a function ... Perl algorithms tend to exchange RAM for speed in most cases ...
    (comp.lang.perl.misc)
  • Re: Steve Jobs demos Macintosh in 1984
    ... Woody wrote: ... Stop annoying the fucking hell out of me, ... For sure not much computer memory had been made in 1948 - but it could ... For sure you got a bit more RAM to use in the Speccy - ...
    (uk.comp.sys.mac)