Re: looping eating 100 per cpu
From: André Nęss (andrena.spamreallysucks_at_ifi.uio.no)
Date: 12/04/03
- Next message: Mattias Campe: "Image and text: howto see the difference"
- Previous message: Spam sucks: "Re: image into variable or database"
- In reply to: Juha Suni: "Re: looping eating 100 per cpu"
- Next in thread: Shawn Wilson: "Re: looping eating 100 per cpu"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Thu, 04 Dec 2003 15:59:18 +0000
Juha Suni:
> kaptain kernel wrote:
>> i've got a while loop thats iterating through a text file and pumping
>> the contents into a database. the file is quite large (over 150mb).
>>
>> the looping causes my CPU load to race up to 100 per cent. Even if i
>> remove the mysql insert query and just loop through the file , it
>> still hits 100 per cent CPU. This has the knock on effect of slowing
>> my script down so that mysql inserts are occuring every 1/2 second or
>> so.
>>
>>
>>
>> here's my script:
>>
>> $fd=fopen("bigtextfile.txt","r");
>>
>> while (!feof($fd) )
>> {
>> $line=fgets($fd,4096);
>> $linecontents=explode(":",$line);
>> mysql_query("INSERT INTO MyDatabase VALUES
>> '','$linecontents[0]','$linecontents[1]'");
>> }
>
> You should try using the built-in functions of PHP to simplify this,
> could propably speed up the process quite a bit. (see
> http://www.php.net/file). I am assuming that doing it your way means
> reading the file _very_ many times, whereas with the example below the
> whole file is read once, and then processed line by line from memory.
Ehm... that means reading a 150MB file into memory, even though you don't
need it there. Not a very good idea! His code only reads the file once, and
only 4KB a time, so there is very little wasted memory.
André Nęss
- Next message: Mattias Campe: "Image and text: howto see the difference"
- Previous message: Spam sucks: "Re: image into variable or database"
- In reply to: Juha Suni: "Re: looping eating 100 per cpu"
- Next in thread: Shawn Wilson: "Re: looping eating 100 per cpu"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
