Re: Asynchronous forking(?) processes on Windows

Hi Xho

I thought you'd be tired of me by now ;) You have good questions. I'm
about to try and answer them within the comments below. In general,
the "harness" I'm in the midst of writing is a way to drive tests on a
remote clients. As you indicated, the clients/slaves will spawn off
child processes which will do the actual work. We want to have the
ability to initiate different commands, shell script, C programs or
whatever on these spawn processes which will then load our Perforce
server. Kinda like Mstone, if you know what that is. If it was just
the test execution "times" on the children my life would be easier.
I'm trying to design the harness to handle the worse case scenario for
tests that have yet been written. which would be handing back the
begin time, the child process output data, and end time. The
discussion continues below...


On Jul 5, 1:52 pm, xhos...@xxxxxxxxx wrote:
Tim <tbra...@xxxxxxxxxxxx> wrote:
Hi again

I appreciate all the help. I hate to resurrect this issue but my
manager re-clarified the requirements of this project and I'm back.
Just to give you an overview of what I'm trying accomplish... I'm
creating a harness for performance testing.

If you only care about *testing* the performance, I would probably use a
completely different method, depending on what aspect of performance you
are testing. If you are only interested in how long it takes the child
to finish, then forget the pipes. Just have the child exit when it is
done, throwing away any output it would have generated in a non-test
environment.


If I could only wish. Sure, the "times" are my primary interest
however the potential for requiring the output of the children is
inevitable. It looks like I'll need the pipes. It's either that or
creating temp files to store the STDOUT of the child processes and
then picking it up later. I already prototyped the temp file idea on
Windows using the Win32::process::create module and dup'ed STDOUT
filehandles. That would be my last resort

A server/master script
will send command requests via TCP to clients/slave daemons. These
slaves will spawn/fork off "n" number of these commands which need run
and finish at their own rate.

Whose own rate? One master talks to several daemons, and each daemon
forks several processes. Is it the daemon's own rate, or the process's
own rate, that is required? We've seen code that reflects the process
communicating back to the daemon, but how/what does the daemon communicate
back to the master?

The individual rate of each spawn process. The master sends off a TCP
request to the client daemon and leaves the connection open for a
response. The daemon spawns off the children which runs the tests.
Upon completion, the daemon sends a return TCP response to the master
with the status of the children's test (times? data?).


The output of these commands needs to
get stored in some fashion.

Unless the point of the test is to test how long it takes to store the
results, why do the results need to be stored?

I think this is explained above.


Although Xho's "foreach" solution cured my problem, the reaping of the
pids is performed in a strict order (reading them until they're
finished). I understand the need for this in order to prevent a
deadlock but my ultimate goal to to keep these processes as
asynchronous as possible (using wait?). Charles' comment on the
"asynchronous wait" seems like what I need but this solution needs to
run on Windows.

Asynchronous waiting won't solve the deadlock issue. It only lets your
parent program do other things while it is waiting (which will be forever
in a deadlock). But if the parent has nothing to do while it is waiting,
there is no point.

Yeah, I just found this out, The asynchronous wait *is* available for
Windows (they say) but it just hung on me.



You could use IO::Select (I've never used it on Windows) to determine which
child is ready to be read at any given time. Once you have read a child's
pipe upto the eof, then you can wait for that child (possibly
asynchronously).

When I first started this project I "borrowed" a chunk of code from:

http://www.wellho.net/solutions/perl-controlling-multiple-asynchronous-processes-in-perl.html

which utilizes select and signals. It worked great on UNIX but failed
on Windows. It looks like the USR1 signals weren't happening on
Windows. :(



Here's my question about an alternative... let's say I keep all the
processing of these children in the children and don't pipe anything
back to the parent. Instead I store the results of the children in a
global/shared hash. Can a single hash be operated on by several
children.

There are modules that allow that to happen (like forks::shared), but I
generally try to stay away from them, especially when I am concerned about
performance.


I need to stick with resident perl modules. We don't want the
requirement of installing additional packages unless it's pure perl
and instantly portable. I'm not asking for miracles at this point. As
a matter of fact, just having the opportunity to tell someone like
yourself where things are at at helped me think this through. If you
have any other ideas I'm ready to listen. Thanks Xho

Tim



.

Relevant Pages

  • Re: Reaping a process without a parent process/service/daemon
    ... Thanks for the replies...you guys are right...I sort of realized that while I ... UNIX/Linux to Windows. ... We have a daemon process that runs in the background and polls a database. ... execs a child process that carries out some specified behavior. ...
    (microsoft.public.vc.language)
  • Re: Unix auditing tools - Windows based.
    ... Cenzic offeres nessus daemon for windows v2.x ... Tennable just released Nessus3 for win32 (basicly Newt thrown ...
    (Pen-Test)
  • Re: Reaping a process without a parent process/service/daemon
    ... You don't need a parent-child relationship as long as you ... UNIX/Linux to Windows. ... We have a daemon process that runs in the background and polls a database. ... execs a child process that carries out some specified behavior. ...
    (microsoft.public.vc.language)
  • Re: Starting an external process from C++
    ... application is a daemon which can be communicated with and writes a logfile. ... My plan is to start up this 2nd application as a child process, ... So even though the 2nd application writes to stdout I don't want to see it. ... As I found out popen will do ok except that I cannot send a kill signal to ...
    (comp.unix.programmer)
  • Re: Asynchronous forking(?) processes on Windows
    ... If you are only interested in how long it takes the child ... One master talks to several daemons, ... Asynchronous waiting won't solve the deadlock issue. ... But if the parent has nothing to do while it is waiting, ...
    (comp.lang.perl.misc)