Re: speeding up cgi perl
- From: skeldoy <skeldoy@xxxxxxxxx>
- Date: Wed, 17 Dec 2008 16:11:45 -0800 (PST)
Isn't dupechecking more a back-office thing best done from the command
line, not CGI?
I know. I got some very particular non-technical customers that wants
to oversee the entire process to spend time. I think the bureaucrats
have too little work and too much time on their hands. But hey! I
can't really tell them what THEY want to do, right? :)
What does that mean? $|=1? mysql_use_result?
$|=1 on everything, everywhere. The mysql doesn't really buffer output
anymore, but the whole thing is in memory - so it is, as I said,
probably not the culprit.
I wouldn't know how to identify *all* sources of buffering present in
a complex system, let alone know how to disable them completely.
I know, but I thought there might be something trivial that I didn't
know about. Good to hear though.
It takes less than a second to stream the entire <big_num> of data
to your screen on the mysql-console? Or do you somehow tell it to
run the query but suppress the actual results?
If i render it on screen from the mysql - I get bottlenecked by the
graphics-adapter/Xorg. When I pipe it to a file it gets done in like a
couple of seconds, and I figure that is how long it really does to do
the query.
Does the CGI have to be invoked by POST, or will GET work?
I use GET.
Use wget to fetch the page. See how long it takes when the results just
have to fetched and stored and not rendered.
I will.
And once you do that and have the results stored as a static file, you
might as well put that in a location that is served up via http and open it
with the client browser. See how long does it take to pull the data over
the net and render it in a browser when it is coming from a static file
rather than a CGI.
The next step. I agree. I plainly just have to be more thorough.. See
your pointers there.
You can try adding -d:DProf or -d:SmallProf (or you favorite profiler) to
the CGI shebang line. Or come up with an adapter run the CGI in a
command-line environment, and profile it from there.
Yeah. Mister Peter mentioned a Devel::NYTProf that I intend to run.
But I will definatly try out DProf and SmallProf like you say.
Glad some of you actually know about these things (like profiling and
such other stuff that I am totally blind to).
On most modern computers, 50MB is a rounding error. Is that the case for
your computer?
? huh ? Is this a new feature of the VonNeumann machine that slipped
my attention?
*Anything* can be done incorrectly, in which case there are things to be
aware of. Are you using multiple MySQL connections in your CGI? If so,
how are you doing it?
doing multiple "my $db1 = the connection stuff ; my $db2 = the
connection stuff ; my $db...
I open them all up in the beginning of the code and keep them over the
length of the run.
Strange thing is that when I parse the data into the DB I do it in the
exactly same manner and the
multiple connections speed that up. Can't really see how multiple
connections would suddenly be an issue,
but strange things happen in the Perl-world, so I have all
possibilities open.
Profile, profile, profile. If that doesn't work, comment out chunks
of code and see if that affects speed (for example, do all the work except
for the actual printing of the html table.)
Again with the profiling. I will man. Thanks for that. Never actually
been a profiling man. Just code and it normally works the way I
intended. Will definelty try it out now..
If the CGI script itself is using 95% of a CPU, then it probably isn't
a browser or network issue. Unless your CPU utilization reporting tool
accounts for I/O waits as CPU usage (I've seen some that do).
I don't really know. All I have done is run top on the linux-server
that runs it.
Not really into the whole userland thing - can it be an apache issue??
15,000 records is not huge. It is just barely even medium. Unless you
have giant blob or text fields. If you do have a large database with
DBD::mysql, it is important to know the benefits and drawbacks of
mysql_use_result.
I know 15000 is supposed to be nada. But 15000 is the magic number
where things tend to slow down.
You must excuse me, I don't normally type English, so I may tend to
write awkwardly.
I will read up on mysql_use_result though. Thanks for the tip. I see
there is much more docmentation to dive into.
Thanks for all the tips and pointers and not least your time!
.
- References:
- speeding up cgi perl
- From: skeldoy
- Re: speeding up cgi perl
- From: xhoster
- speeding up cgi perl
- Prev by Date: Re: speeding up cgi perl
- Next by Date: Re: mathematical operation in a perl one liner in substitute
- Previous by thread: Re: speeding up cgi perl
- Next by thread: Re: speeding up cgi perl
- Index(es):
Relevant Pages
|
