Re: Speeding up an application - general rules


I haven't yet done any measuring of the CPU usage for the processes,
but will look into that -TX. I just heard yesterday (the day of my
post) that the application was bogging down. When I do my testing, I'm
working with live, production data, but typically limit my search to
one of three patterns and do it on only one or two directories. I want
to get my results back quickly. The users of this app apparently make
heavy use of this and are looking for the "needle in a haystack".

On Dec 22, 1:26 am, xhos...@xxxxxxxxx wrote:
"Petyr David" <phyn...@xxxxxxxxx> wrote:
Basically: the script uses perl's system command to run a long winded
"find" command which is piped to sed to correct patterns that match
HTML markers. The matching lines are then shoved into an array. The
elements of the array are moved into a hash for the purpose of sorting
the file names. Then file names and matching lines are printed.

Q: Can I speed things by eliminating the sed command and letting Perl
filter and modify the matching patterns?Probably not. It should be a 30 second job to take out the sed pipe.
Sure, the answers will now be wrong, but unless it gives the wrong answers
much faster than it used to, you will know there is no speed benefit to be
had by rewriting the sed into Perl.

If so, how much of a
performance gain?

Is using Perl's grep to search through every file for the pattern
faster than using the find command?Probably not. Also, Perl's grep (currently) forces the list to be
evaluated to completion (in memory) before it gets started, so potentially
takes much more memory. You may want to look at Perl's File::Find,
although I see no particular reason to think it will be faster than the
system's find.

The find command has the advantage
that I can search for files of a certain date rather easily. Again:
could that be done more rapidly by Perl's looking at the file's mod
time?Probably not more rapidly, no.

What is the total CPU usage? What is the relative usage of each process
(perl, find, sed)?

Xho

--
--------------------http://NewsReader.Com/--------------------
Usenet Newsgroup Service $9.95/Month 30GB

.

Relevant Pages

  • Re: Error Using Remoting, System.OutOfMemoryException anyone ever seen this?
    ... it to transfer the DataSet to the client like I wanted. ... >> As Integer, ByVal pnUserID As Integer, ByVal piView As Short) As DataSet ... >> As for memory use, the client which is hung is not peaked out of memory, ... >> it is running 1% cpu usage and barely any memory used. ...
    (microsoft.public.dotnet.framework.remoting)
  • Re: plotting cpu and mem consumption
    ... >>amount of cpu usage and memory consumption of the program. ... > profiling library (or libraries, ...
    (comp.lang.cpp)
  • Re: XP Professional SP@ is slow --- very slow
    ... The second most probable cause of slow performance is simply that the applications you have open, including background processes, require more memory than your computer has to offer. ... Go to the Processes tab and display the processes in decreasing order of CPU usage. ... If your computer is of recent vintage, say, over the last four years, then unless you're running a busy server you won't see a big speed improvement from defragmenting, even with third-party tools. ... The only thing I can think of that would cause such a CPU usage is a background program runnibg somewhere, ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: Problems with newly installed Windows XP Home
    ... No obvious process hogging memory or CPU usage but only ...
    (microsoft.public.windowsxp.setup_deployment)
  • TaskInfo 8.1.0
    ... including System threads. ... it shows CPU usage, scheduling, path, open files, memory, DLLs, ... CPU, Memory, Cache usage, Data rates for Disk, ... Download URL: http://www.iarsn.com/taskinfo/tskinf81.exe ...
    (comp.software.shareware.announce)