Re: perl multithreading performance

On Aug 27, 5:06 pm, Ted Zlatanov <t...@xxxxxxxxxxxx> wrote:
You may be hitting the limits of I/O.  Try feeding your script
pre-canned data from memory in a loop and see if that improves
performance.

No, the IO is fine - there are pretty much always $q_in->pending > 1,
and as the script does its thing, number of pending buffers sometimes
goes beyond 10.

 It also depends on what kind of processing you are doing
on input lines.

Just trying to match multiple regexps against each line.

Also, check out the swatch log file monitor, it may do what you need
already.

Nope, it doesn't :( I already have the single-threaded script, which
has been working for years now, but the amount of logs it needs to
process keeps growing, and I'm basically at the point where it can
only keep up with the speed with which logs are being written, so if
there's back-log for whatever reason - it might not catch up, so I'm
looking into how I can improve its performance.
.

Relevant Pages

  • Re: [opensuse] howto - Read Data from Serial Port?
    ... A linux question. ... I have a phone system that will output call logs through an rs-232 cable. ... I would like to write a script that will read the data from the serial port and input it into a mysql table. ... Another question, If I write a small script and "sleep" for 30 seconds or so during the loop to cut down on racing the loop, will that have any unintended consequences I should look out for? ...
    (SuSE)
  • Re: Report in FM 6 or 7
    ... achieve this report format will have to be done via scripting. ... but they will simplify the script and Sort Order. ... End Loop ... The normal 'Dancer' field is used so that if there are more than one ...
    (comp.databases.filemaker)
  • Re: recreate database script not work
    ... I got a script which is supposed to regenerate database systax. ... REM gen_dbse_9.sql ... end loop; ...
    (comp.databases.oracle.server)
  • RE: SP1 Install Errors
    ... I have received your logs files and emails. ... time difference between us, and as you know, SP1 log is a large file and we ... First attempt at executing script sp1_serv_uni.sql failed; ... Microsoft CSS Online Newsgroup Support ...
    (microsoft.public.windows.server.sbs)
  • RE: Domain Startup Scripting
    ... the following command always works and logs the appropriate info ... above which tells me it is accessing the script, ... could help me with troubleshooting a domain startup script routine. ... @echo off ...
    (microsoft.public.windows.server.scripting)