huge file : parsing problem
- From: saurabhsinghvi@xxxxxxxxx (Saurabh Singhvi)
- Date: Wed, 29 Mar 2006 15:53:09 +0000
HI there,
I have a huge file (~7GB) of text data. I need to parse it to extract
something : basically minima and maxima, and then i need to quantify
the data into an output file based on ranges.
Now the problem is that the traditional method of opening and foreach
which is as follows:
open(FILE,"data.txt");
$min = 100;
$max = 0;
foreach $data (<FILE>) {
if ($min > $data ) {
$min = $data ;
}
elsif ($max < $data) {
$max = $data ;
}
}
print $max, $min;
close FILE;
fails to work, as perl is trying to load the complete file into memory!
Now i also tried changing it to STDIN instead, by catting the data from
the text file.
../script.pl < data.txt
but this too gave the same error
Out of memory!
So kindly help me on how to do this thing.
thanks
Saurabh
- Follow-Ups:
- Re: huge file : parsing problem
- From: Mr. Shawn H. Corey
- Re: huge file : parsing problem
- Prev by Date: Re: Sub coding question...
- Next by Date: Re: huge file : parsing problem
- Previous by thread: SOAP::Lite perl client and .net webservice method.
- Next by thread: Re: huge file : parsing problem
- Index(es):
Relevant Pages
|
