Re: perl multithreading performance

Leon Timmermans <fawaka@xxxxxxxxx> wrote:
On Wed, 27 Aug 2008 23:53:09 +0200, Martijn Lievaart wrote:

Perl threading, well frankly, sucks. You may want to switch to another
language with re support that meets your needs.

Some would say all threading sucks. All approaches are either hard to get
a proper performance from or hard to get correct. At least the queue
approach perl promotes gets one of them right.

Also lets not forget that Perl at least supports preemptive threading.
Ruby doesn't at all and python has a giant interpreter lock, making it
useless for this kind of problem.

I fleshed out the OPs example code to make it runnable, using a simple
foreach (1..400) {}; to simulate the processing of each line in the
consumer threads (400 because that is what provided a throughput of 30_000
per second in a simple non-threaded model) and was pleasantly surprised.
I got a substantial speed up by using threading, with a factor of 3
improvement in throughput by using $cpu_count=4 (4 consumer threads, plus
main thread).

I still wouldn't use threads on my own code for something like this,
though. I'd just start 4 processes assigning each a different chunk of the
data.


Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
.

Relevant Pages

  • Re: [PATCH 1/3] Separate IRQ-stacks from 4K-stacks option
    ... >>the irq threading must have a cost, ... different resources at the same time, keeping more resources busy. ... this can increase overall system throughput significantly. ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)
  • Re: Best way to design multithreading application
    ... threads are working on disparate resources. ... That being said I don't know if you'll see the performance boost you're looking for. ... Since a given link in the chain can't do anything with the data until the previous link is done with it, threading isn't going to improve throughput. ...
    (microsoft.public.dotnet.framework)
Loading