Re: spawning pyhon apps...

On Jan 9, 4:07 pm, "bruce" <bedoug...@xxxxxxxxxxxxx> wrote:
thanks jason....

or i could also, simply iterate through a loop of the names of the "child
processes" i want to spawn, using the names in the subprocess.popen, and
then proceeding with the rest of your example..

question though... if i have a child app that's hanging.. how do i kill it.

or is this getting into the aspect of using interprocess pipes, where if the
parent isn't getting a 'live ping' via the pipe back from the child after a
certain amount of time... it could kill the child...

thoughts/comments...

thanks

-----Original Message-----
From: python-list-bounces+bedouglas=earthlink....@xxxxxxxxxx

[mailto:python-list-bounces+bedouglas=earthlink....@xxxxxxxxxx]On Behalf
Of Jason Scheirer
Sent: Friday, January 09, 2009 3:59 PM
To: python-l...@xxxxxxxxxx
Subject: Re: spawning pyhon apps...

On Jan 9, 3:43 pm, "bruce" <bedoug...@xxxxxxxxxxxxx> wrote:
hi jason....

forgive me... but in your sample:
        my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
        %i.txt"%x) for x in xrange(10)]
are you spawning 'foo.py' 10 times? that can't be right!
so just what is "foo.py" used for? what am i missing...

it looks like the my_popenobjects array is iterated through to check the
statuscode. is the statuscode the value that would be returned from a
child
python script via something like "return(2)"....

i've seen mention of os.waitpid(..) does this play into waiting for child
processes to complete, or determine if they've terminated??

thanks

-----Original Message-----
From: python-list-bounces+bedouglas=earthlink....@xxxxxxxxxx

[mailto:python-list-bounces+bedouglas=earthlink....@xxxxxxxxxx]On Behalf
Of Jason Scheirer
Sent: Friday, January 09, 2009 3:19 PM
To: python-l...@xxxxxxxxxx
Subject: Re: spawning pyhon apps...

On Jan 9, 2:47 pm, "bruce" <bedoug...@xxxxxxxxxxxxx> wrote:
hi...

toying with an idea.. trying to figure out a good/best way to spawn
multiple
python scripts from a parent python app. i'm trying to figure out how to
determine when all child apps have completed, or to possibly determine
if
any of the child processes have died/halted..

parent app
 spawn child1
 spawn child2
 spawn child3
 .
 .
 .
 spawn childn

do i iterate through a os.waitpid(pid) for each pid of the child
processes
i
create?

is there another approach? code samples/tutorial...??

i've seen various approaches via google, but not just what i'm looking
for..

thanks

Investigate the subprocess module, you probably want Popen objects.
You can do a poll loop like

my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
%i.txt"%x) for x in xrange(10)]

while any(popenobject.returncode is None for popenobject in
my_popenobjects):
  time.sleep(0.25)

If your tasks are more like function calls and less like shell
scripts, then investigate writing your python as an importable module
and use the multiprocessing module, which will do threading/
subprocessing for you.
--http://mail.python.org/mailman/listinfo/python-list

Correction: statuscode is wrong. It's returncode.

Foo.py is the hypothetical Python script you want to run in a
subprocess. In this example, I have an external shell script named
foo.py, and I am indeed spawning 10 copies of it, each with a second
argument that varies (foo.py --filename=file0.txt, foo.py --
filename=file1.txt, ... foo.py --filename=file9.txt). You don't need
os.waitpid() with a Popen object, there is a Popen.wait() method you
can call which will accomplish the exact same thing. I'm polling the
Popen.returncode for each process' return code (which is the numeric
code a process returns when it finishes, like sys.exit(x) or return,
or gives None if it's not done yet. What this sample is doing is
opening 10 copies of the script and running them in parallel, if you
want to run it is serial then you can do a simple for loop and .wait()
on each:

for cmd in ('a', 'b', 'c'):
  sp = subprocess.Popen(cmd)
  sp.wait()
  print "Command %r completed with status %i" % (cmd, sp.returncode)

I'm still not 100% sure what you're trying to accomplish. What is the
exact problem you are wishing to solve?
--http://mail.python.org/mailman/listinfo/python-list



Yes, so to open your processes you can loop over a list of commands
and create new subprocess.Popen(cmd) objects for each.

Everything is explained:

http://docs.python.org/library/subprocess.html#popen-objects

You can do .terminate() to kill a process that may be hanging, you can
get the .stdout and poll on its .read() to see if it's still putting
anything out to the console.
.

Relevant Pages

  • Re: trying to understand fork and wait
    ... old habits based on learning to script in REXX on the ... > the child reads it. ... situation for me (drop through to bottom/go back to top of loop). ... just to keep a hold of the exit code. ...
    (comp.lang.perl.misc)
  • RE: spawning pyhon apps...
    ... if i have a child app that's hanging.. ... Subject: spawning pyhon apps... ... You can do a poll loop like ... I have an external shell script named ...
    (comp.lang.python)
  • (another) datarelation datagrid question
    ... in a manner described best by "How to Display Child Table Rows as a Column ... that i would like bound to the final grid. ... inside that loop i loop thru the child table and add the data from the ... look exactly as i want and as a bonus it filters parent records without ...
    (microsoft.public.dotnet.framework.adonet)
  • Re: File handle re-use woes using pipe within a loop
    ... > child will die before the parent has read the output from it. ... > definitions within the while loop. ... The whole point of 'my' variables is that when they go out of scope ...
    (comp.lang.perl.misc)
  • TCP Server+perl
    ... Client each time a new connection is established at the Socket. ... each child completes processing it closes/ends. ... # Main loop control variable ... You exit the ...
    (comp.lang.perl.misc)