Re: PHP CLI & Forking children

qwertycat@xxxxxxxxxxxxxx wrote:
On Sep 29, 7:51 pm, Jerry Stuckle <jstuck...@xxxxxxxxxxxxx> wrote:
Just wondering - why do you need to fork processes, anyway? There's a
lot of overhead in doing it, and if they're all CPU bound anyway you
aren't going to gain anything (unless you have a potload of CPU's).

Forking is good if you have different processes using different
resources. But when they have to contend for the same resource,
performance often goes down.

Instead of writing a PHP script that downloads 2 million headers from
a newsgroup in a single connection (which will cause PHP to crash
anyway as it'll reach 500MB+ memory usage), I thought it would be
better to launch 4 processes do download it in chunks of 50,000
headers - with 4 connections to the same NNTP server.


Which means you'll be downloading 500MB+ anyway - just in different processes.

Or you could get some headers and cache them to disk, processing them later.

But which newsgroup has 2M+ headers? Glad I don't have to read that one! :-)

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@xxxxxxxxxxxxx
==================
.

Relevant Pages

  • Re: OE6 newsgroup reader problem...
    ... every time you open the newsgroup, ... Now return to "Get XXX headers at a time" and uncheck the option completely; ... unless you copied them to another folder for archiving. ... at a time", that is, to download all the headers again. ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)
  • Re: Newsgroup problem
    ... Setting up Outlook Express Newsreader ... Remove Get XXX headers and click on this link and see if all messages download. ... To mark an individual message as read or unread, ... To mark all newsgroup messages as read when you quit Outlook Express or go to another newsgroup, on the Tools menu, click Options. ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)
  • Re: OE Is Deleting My NG Headers
    ... You can get situations where headers appear and when you go to get the ... remains until the newsgroup is next synchronised. ... Compacting was supposed to work in the ... I was playing aroudn with the synchronization settings, ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)
  • Re: OE Is Deleting My NG Headers
    ... newsgroup folders can easily be replaced. ... The delete option when compacting was also a common cause of file ... I don't have any problem getting headers. ... I was playing aroudn with the synchronization settings, ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)
  • Re: Not sure: Removing headers that no longer reside on server
    ... > /Every/ newsgroup exhibits the following behavior: ... > the retention for that particular server). ... >> duplicate messages or headers that don't delete. ... >>> Also, I have turned off background compacting, because of bugs, at ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)