Q: How to improve CGI performance for this?

Hello,

I'm not sure that this is the right group for this question - Googling
for similar questions gave me always "comp.lang.perl.misc", but if this
is OT here, please, let me know.

The problem I'm having is with a CGI script that needs to load and
parse some data from a quite large file, then do some extra processing
based on user form input and return. The slow part is (I guess)
loading and parsing the data from the file. Although this is taking a
fraction of a second, the script is quite often invoked by a number of
customers simultaneosly, and is causing problems with my shared hosting
provider (reportedly 25% CPU load at some times).

I'm looking for high-level ideas for a solution. What I can come up
with is:

1) Optimizing the file format for easier parsing - not much could be
done here, this is pretty straightforward task. The file size is about
200K, and there is simply no way to avoid loading it. Partially loading
also doesn't work (splitting it to pieces, things like this).

2) Switching to PHP - this means rewriting everything, what a mess...

3) Using something like FastCGI, providing that the hosting provider
have it. Do you think this can help? I don't know much about FastCGI,
can I somehow preload the data in memory and just use it from the
script?

4) Using mod_perl? Don't know much about it too. Can I parse the data
from the file once, have it stored in memory and each time my script is
invoked access it? How much cooperation is required from my provider
for this?

5) Do something else?

Any help/ideas/suggestions are appreciated.

Thanks,

- Alex

.

Relevant Pages

  • Re: Problem with ASP on Webserver
    ... No, I know the memory model, I have made my own Virtual memory dump program ... function in the script ... (Accept-Language:) ...
    (microsoft.public.windowsce.platbuilder)
  • Re: Problem with ASP on Webserver
    ... Entrek's TOOLBOX is a lot better for memory stuff. ... function in the script ... (Accept-Language:) ...
    (microsoft.public.windowsce.platbuilder)
  • Critique my code: using Lisp to parse lots of "ps -elf" files in AIX
    ... It's my first use of Lisp at work, ... and another ksh script to analyze the output. ... success using them to diagnose some of their own memory problems, ... an hour to process about an hour's worth of data (i.e. 60 data files). ...
    (comp.lang.lisp)
  • Re: JSON without AJAX
    ... - "loading scripts" after the document was loaded is still unreliable. ... this script is perfectly reliable. ... unreliablity of dynamically loading any kind of content ... They are an invention based on a glue invented some 80 years ...
    (comp.lang.javascript)
  • Re: Incude statement - is there such a thing?
    ... I can't imagine any memory usage issues. ... And it that WSF references ... other script files (as the Docs often suggest to be ... > you had 50 statements that you wanted to execute 1000 times in a loop. ...
    (microsoft.public.scripting.vbscript)