Re: optimize log parsing

"it_says_BALLS_on_your forehead" <simon.chao@xxxxxxx> wrote:
....
> >
> >
> > Don't think in terms of processes. If you're using processes for that
> > kind of thing you'll need to find a way for them to communicate
> > (possibly pipes, or maybe shared memory). Threads takes this work off
> > your shoulders as they can share data in a simple and secure manner.
> >
>
> Tassilo, thank you very much for your help. If i could trouble you once
> more for your insight...
>
> What benefit does the thread model have that the following does not?
> What drawbacks?
>
> use Parallel::ForkManager;
> $pm = new Parallel::ForkManager($MAX_PROCESSES);
> foreach $data (@all_data) {
> # Forks and returns the pid for the child:
> my $pid = $pm->start and next;
> ... do some work with $data in the child process ...
> $pm->finish; # Terminates the child process
> }


The ForkManager method does not allow the children to report back
to the parent in a meaningful way (i.e. beyond exit status). Threads do
readily allow that (although Tassilo's specific example doesn't).

There is probably more overhead in forking a process for each task than
there is in popping from a shared queue. Unless you have many thousands
of very very short tasks, this difference is probably negligible.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
.