out of memory

I am running out of memory when I try to select about 100K rows. Here is
some code:

my $cel_sth = $dbh_from->prepare($sqlstring);
$cel_sth->execute();
my @rowvalues;
my $rv;
my $elementresult_id;
my $element_id;
my $quantification_id;
my $signal;

$rv = $cel_sth->bind_columns(\($elementresult_id, $element_id,
$quantification_id, $signal));

while($cel_sth->fetch()){
@rowvalues = ($elementresult_id, $element_id, $quantification_id,
$signal);
my $printstring= "insert into $table_name values(";
foreach my $column (@rowvalues){
if (defined $column){
if($column =~ /\D/){
$column = "'".$column."'"
unless ($column =~ /\d*\.\d+/);
}
$printstring.= "$column,";
} else {
$printstring.= "NULL,";
}
}
$printstring =~ s/,$/);/;
print "$printstring\n";
}

I guess the problem is it tries to hold all the rows in memory. Is there a
way to just get 1 or a few rows at a time?
--
Lou Fridkis
Human Genetics
57920

Relevant Pages

  • Re: out of memory
    ... wouldn't it just be better to switch back ... On 12/29/06, Ron Savage wrote: ... That saves copying the info into @rowvalues. ... The reason I think it tries to hold all the rows in memory ...
    (perl.dbi.users)
  • Re: out of memory
    ... dirty to generate some test data. ... since I am eager to improve. ... That saves copying the info into @rowvalues. ... The reason I think it tries to hold all the rows in memory ...
    (perl.dbi.users)
  • Re: out of memory
    ... Assuming you're using mysql, take a look at the mysql_use_result attribute. ... my @rowvalues; ... I guess the problem is it tries to hold all the rows in memory. ... Lou Fridkis ...
    (perl.dbi.users)
  • Re: "Out of memory!" ???
    ... if there is enough memory to ... process the array in a loop by using "foreach". ... You make heavy use symbolic references. ... each iteration of the outer most foreach loop, ...
    (comp.lang.perl.misc)
  • Re: huge file : parsing problem
    ... basically minima and maxima, and then i need to quantify ... Now the problem is that the traditional method of opening and foreach ... as perl is trying to load the complete file into memory! ... Perl tutorials at http://perlmonks.org/?node=Tutorials ...
    (perl.beginners)