Re: fork,exec, and parallel processing
- From: xhoster@xxxxxxxxx
- Date: 26 Mar 2007 16:21:03 GMT
"drew@xxxxxxxxxxx" <drew@xxxxxxxxxxx> wrote:
Hi,
I've a need for a script that can query multiple NetBackup servers for
a clientlist, in parallel (rather than sequentially) if possible.
Did you look for existing modules that will do that?
I've developed the following code, which seems to work, but also seems
to only run sequentially. What can I do to correct the script so it
will run the children in parallel?
Thanks,
Drew
__BEGIN_SCRIPT__
use strict;
my $passes = 0;
my @kids = ();
my @clients;
my %backup_results = ();
my @serverlist = qw/server1 server2/;
my %client_to_master = ();
for my $server (@serverlist) {
my $pid = open(KID_TO_READ,"-|");
You should use lexical file handles.
my $pid = open(my $fh, "-|") or die;
push @handles,$fh;
### or maybe, depending on your needs:
### $ioselect->add($fh);
print "PID: $pid\n";
if ($pid) {
# parent
@clients = ();
while(<KID_TO_READ>) {
next if (/Windows/);
next if (/^Hardware /);
next if (/^-----/);
my ($hardware,$os,$client_name) = split;
push(@clients,$client_name);
$client_to_master{$client_name} = $server;
}
close(KID_TO_READ) or warn "Child exited: $?\n";
$backup_results{$server}{'clientlist'} = [@clients];
push(@kids, $pid);
This should all be moved to after the the parallel open loop. I don't
think you need to save the $pid, only the file-handle.
} elsif ($pid == 0) {
# child
my @options = qw/ssh/;
my $path = '/usr/openv/netbackup/bin/admincmd';
my $cmd = 'bpplclients';
my $bpplclients = $path . '/' . $cmd;
push(@options,$server,$bpplclients);
exec('/usr/bin/sudo',@options) or die "Can't exec: $!
\n";
exit(0);
} else {
die "Can't fork: $!\n";
}
}
This part would be pretty much the same.
foreach(@kids) {
waitpid($_,0);
}
I don't think this currently does anything, as the close of a pipe open
automatically does a waitpid.
Anyway, you need to put the above parent code here. The exact nature
depends on what it is you need to do, as "parallel" covers a vast range
of options. If the children can be harvested in an arbitrary order, then
just use a:
foreach my $handle (@handles) {
## harvesting code from above here, with suitable changes.
}
If a fast child gets stuck behind a slow one in the @handles list, then
it will just around idle until the slow one is done, before it can get
harvested. Often, that is not a problem.
On the other hand, if the children must be harvested ASAP after each one is
done, then you would need to use IO::Select or something like it.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
.
- References:
- fork,exec, and parallel processing
- From: drew@xxxxxxxxxxx
- fork,exec, and parallel processing
- Prev by Date: fork,exec, and parallel processing
- Next by Date: Re: order of evaluation
- Previous by thread: fork,exec, and parallel processing
- Next by thread: Re: fork,exec, and parallel processing
- Index(es):
Relevant Pages
|
Loading
