Re: Processing workload distribution
- From: Ben Morrow <ben@xxxxxxxxxxxx>
- Date: Mon, 25 Feb 2008 17:14:55 +0000
Quoth Ted <r.ted.byers@xxxxxxxxxx>:
The machine is a single processor/quad core server running the latest
Windows Server.
The Perl we're using is the 64 bit build of 5.8.8 from Activestate.
I created a script that uses threads to launch a series of standalone
SQL scripts (I use system to invoke mysql to run my SQL scripts).
In general it is easier to talk to the database directly using DBI than
to launch external mysql processes. In this case, however, you may find
it easier to make things run in parallel with an external command.
I had thought that system was spawning child processes, but it seems
that the script that launches them waits on each child before
launching the next.
Yes, this is how system works. As a special case, on Windows only,
calling
my $pid = system 1, "mysql ...";
(with a literal 1 as the first argument) will spawn a new process and
*not* wait for it. You can use the returned pid as an argument to wait
or waitpid. Alternatively, for more control, you can use Win32::Process
or IPC::Run.
It is looking like the threads launches all without wating for the
next, which is closer to what I want.
You would have to post your code for us to see what is happening here. I
think probably each thread is launching one child and waiting for it,
but since you have several threads you have several children.
However, regardless of what I have tried so far, all the hard work is
being done by one core, with the other three mostly sitting idle.
Are you *certain* you end up with more than one mysql process running at
a time? You should be able to check easily with Task Manager. If you
don't, then you need to fix that (or switch to using DBI to talk to the
database and Perl threads for parallelism). If you do, yet they are all
running on the same core, then something odd is happening: you will need
to show us a (minimal) script that runs two processes at the same time
that still end up on the same core.
BTW: are there any 64 bit modules on CPAN? The repository for the 64
bit build that Activestate has provided seems to be empty.
Modules on CPAN are (generally-speaking) architecture-neutral, given
that CPAN only holds C and Perl source. If ActiveState don't provide
64-bit ppms, then you can install pure-Perl modules directly from CPAN,
but XS modules will require a copy of the compiler used to build your
perl, probably a 64-bit version of MSVC. You may be able to get gcc
working (the 32-bit version of AS Perl has support for building modules
with gcc), but I don't know how well the 64-bit version is supported.
Ben
.
- References:
- Processing workload distribution
- From: Ted
- Processing workload distribution
- Prev by Date: Re: grep specific info from a file
- Next by Date: Re: Spreadsheet::ParseExcel - How to get certain Cells
- Previous by thread: Processing workload distribution
- Next by thread: Re: Processing workload distribution
- Index(es):
Relevant Pages
|
