Starting Parallel Processes for machine queries.

[erdman]

I have created a script that takes a machine||ip list and then checks to see if particular patches have been installed. Unfortunately, the script currenly is linear. I am wanting to speed up the process by running the query against multiple machines without waiting on a previous machine to finish. Being that the number of machines will be above 7000, i also need to limit the number of queries being done at a single time so that I do not bog down my machine. Any help or suggestions would be greatly appreciated.

Thanks,
ERDMAN

 

[mlibeson]

If you are running in a unix-like environment, then use the fork function. If you are in windows, then I am not sure if the fork function has been successfully ported to windows yet, but you could try. If running in mac, I do not know, but I believe fork should work as there is more unix in macs these days. The fork function will allow you to spawn children that will run indepently. Keep track of your children to make sure they complete and do not start 7000 children all at once. Try to spawn children that process batches of ips instead of just one.


Michael Libeson

 

[erdman]

Fork() is ported for windows with a few caveats. My issue is I have not used fork before and do not quite understand the functionality of the command. I have been unable to find a simple generic example of Fork being utilized for anything.

 

[erdman]

Let me be more specific.

I dont understand how this code:
#!/usr/bin/perl -w

my $pid = 0;
my $forked=0;
my @family;

for ($i = 0; $i<2; $i++){
$pid=fork or warn "Couldn't fork: $!\n";
$forked *= 2;
print "For Loop forked, $forked\n";
print "For Loop pid, $pid\n";


if ($pid) {push @family, $pid;
print "Parent pid, $pid\n"}

else {
@family=();
$forked++;
print "Child printing forked, $forked\n\n\n";}

gets this result:

For Loop forked, 0
Couldn't fork:
For Loop pid, -3280
For Loop forked, 0
Parent pid, -3280
For Loop pid, 0
Child printing forked, 1


For Loop forked, 0
For Loop pid, -2156
Parent pid, -2156
For Loop forked, 2
Couldn't fork:
For Loop pid, -3404
Couldn't fork:
Parent pid, -3404
For Loop forked, 0
For Loop forked, 2
For Loop pid, 0
For Loop pid, 0
Child printing forked, 1
Child printing forked, 3

once again, any and all help is greatly appreciated.

 

[mlibeson]

$pid if not 0 is the child pid not the parent. If $pid is 0, then you are in the parent section of code.

Michael Libeson

 

[Xaqte]

Being a newb, I was having difficulty understanding forking as well. I eventually ended up using Parallel::ForkManager.
I found this a little easier to use/understand until I get a little more experienced. I hope this helps!

X

 

[icrf]

I played with that awhile back and wrote some demo kind of code using fork and threads. Threads can be much more efficient if your version of perl was compiled to support it. Anyway, I don't know if it's blasphemous to link to another technical forum, but here goes:

http://forums.devshed.com/showthread.php?t=132915&page=2&pp=15#post581897

Hope you find some use in it.

________________________________________
Andrew

I work for a gift card company!

 

[MikeLacey]

mlibeson,

I beg your pardon - but fork returns either the PID of the (child) process you just started or zero to tell you that you're *IN* the child process.

If it returns 0 you're the child.

If it returns non-zero you're the parent.

Mike

http://www.google.com/tsunami_relief.html

shows ways to help with Tsunami Relief.

You cannot really appreciate Dilbert unless you've read it in the
original Klingon.

Want great answers to your Tek-Tips questions? Have a look at faq219-2884