From comp.graphics.raytracing Tue Jun  7 15:31:48 1994
Path: uniol!zib-berlin.de!zrz.TU-Berlin.DE!netmbx.de!Germany.EU.net!EU.net!howland.reston.ans.net!gatech!newsxfer.itd.umich.edu!caen!siemens.engin.umich.edu!jmayer
From: jmayer@siemens.engin.umich.edu (Jonathan Mayer)
Newsgroups: comp.graphics.raytracing
Subject: POVRAY+PERL=ANIMATION
Date: 6 Jun 1994 04:42:52 GMT
Organization: University of Michigan Engineering, Ann Arbor
Lines: 660
Distribution: world
Message-ID: <2su9gcINNnf0@srvr1.engin.umich.edu>
NNTP-Posting-Host: siemens.engin.umich.edu
X-Newsreader: TIN [version 1.2 PL2]

Hey folks!

I've got some PERL scripts here that I've developed to help me do animation
using POV-Ray.  My scripts (working together) perform the following:
   - spawns off rendering processes to any number of other processors
   - "collates" the generated images back onto your local machine.
   - automatically creates an MPEG or a directory of GIFs, along with
     a script that runs the animation invoking imagemagick's "animate."
   - pops up a nifty little "wall of xloads" showing you load on the
     machines running your rendering processes.  :)

CAVEATS and KNOWN BUGS: (USE AT YER OWN RISK)
   - if you ain't using Unix, this is all useless to you. (go install Linux
     or something, eh?)
   - the part of my script that's supposed to detect if the remote process
     failed to render a frame correctly (due to either internal or external
     factors) just DOESN'T WORK.  Make dang sure your scene files are
     BULLETPROOF before you run, or else bad things happen.  Also try to
     run your processes on machines where users aren't like to kill 'em.
   - I *really* recommend making a subdir in your /tmp directory as a
     destination "imagedir".  This is *especially* true if you're using
     AFS and have to bollux about with kerberos tokens.
   - Some knowledge of PERL might be necessary to customize the bugger
     for your environment.  
   - requires lots of other tools, including:
	mpeg_encode
	the PBMPLUS package
	the ImageMagick "animate" tool
	POV-Ray ( or Rayshade, if you feel like doing the kustomizing..)
   - I'm still developing this thing as time allows.  I'm only posting it
     in it's current sloppy form because I find it VERY useful.  E-mail me
     if you got problems or if you don't got problems, but NO PROMISES, eh?
   - IMPORTANT!!! REMEMBER TO FIX THE PATHNAMES IN "CONFIG.INC," and also
     fix the pathnames to find CONFIG.INC in ALL the other scripts!!!!!!!
     Remember to fix the #! line to contain a path to *your* perl instead
     of the current "/usr/um/bin/perl", and also remember to make these
     scripts executable (chmod +x) else the spawner'll choke.
   - Sorry, no docs yet (don't need 'em).  Once this evolves to a more
     stable point, maybe I'll release it as a "package."  If you make
     any useful modifications, please send me e-mail and I'll try to 
     consolidate 'em into a truly useful tool.

The scripts are:
	CONFIG.INC - configuration file used by the other scripts.
	spawner - the process spawning "engine."
	pickmach - a script to generate a list of "good" remote machines
		(currently uses NETUSE package ... customize for your site)
	rendgifs - a wrapper around "spawner" that generates your 
		animation as a dir full of .GIFs (1 GIF per frame).
	rendmpeg - a wrapper around "spawner" that compiles the rendered
		frames into an mpeg using mpeg_encode.
	povframe - a wrapper around "POVray" that adds flexibility to
		the package .. invoked for each frame by the "rend..."
		scripts.
	pov - a wrapper around "POVray" similar to povframe ... good for
		testing your scene file before doing a rendmpeg.

CONFIG.INC ---------------------------------------------------- >8
# Configuration file for toolkit written in PERL by
# Jonathan Mayer, 1994.

# PovRay files:
$POVHOME = '/usr/contrib/povray';
$POVBIN = "$POVHOME/bin";
$POVINC = "$POVHOME/include";
$povray = "$POVBIN/povray";

# Toolkit Scripts used by other scripts:
$SCRHOME = "$POVHOME/scripts";
$spawner = "$SCRHOME/spawner";
$povframe = "$SCRHOME/povframe";
$pickmach = "$SCRHOME/pickmach";

# PBMPLUS tools:
$PBMHOME = '/usr/um/pbmplus';
$PBMBIN = "$PBMHOME/bin";
$ppmquant = "$PBMBIN/ppmquant";
$ppmtogif = "$PBMBIN/ppmtogif";
$tgatoppm = "$PBMBIN/tgatoppm";

# IMAGEMAGICK tools:
$IMHOME = '/usr/contrib/imagemagick';
$IMBIN = "$IMHOME/bin";
$animate = "$IMBIN/animate";

# MPEG_ENCODE tools:
$MPEGHOME = '/usr/contrib/povray/mpeg';
$mpeg_encode = "$MPEGHOME/bin/mpeg_encode";

# MVAS utilitites:
$MVASHOME = '/usr/viz';
$MVASBIN = "$MVASHOME/bin";
$mvasrecord = "$MVASBIN/mvasrecord";

# NETUSE utilities:
$hostinfo = '/usr/caen/bin/hostinfo';
$getmach = '/usr/caen/bin/getmach';

# Unix utilities:
$rm = "/bin/rm";
$ls = "/bin/ls";
$rsh = "/usr/ucb/rsh";
$xload = "/usr/openwin/bin/xload";
$cat = "/bin/cat";
$xv = "/usr/um/bin/xv";

# END OF FILE.
end of CONFIG.INC --------------------------------------------- >8


spawner ------------------------------------------------------- >8
#! /usr/um/bin/perl

require '/afs/engin.umich.edu/contrib/povray/scripts/CONFIG.INC';

################### USER CONFIGURABLE STUFF ###########################

@rshOptions = ("-n");

$USE_HOSTINFO = 1;	# true or false (1 or 0)

@MachineList = (
	"ny.engin.umich.edu",
	"pa.engin.umich.edu",
	"wi.engin.umich.edu",
	"ak.engin.umich.edu",
	"ga.engin.umich.edu",
	"al.engin.umich.edu",
	"nv.engin.umich.edu",
	"tn.engin.umich.edu", 
	"wv.engin.umich.edu",
	"ky.engin.umich.edu",
	"va.engin.umich.edu",
	"tx.engin.umich.edu",
	"ms.engin.umich.edu", 
	"ca.engin.umich.edu", 
	"sc.engin.umich.edu", 
	"az.engin.umich.edu", 
	"ia.engin.umich.edu", 
	"me.engin.umich.edu", 
	"ut.engin.umich.edu", 
	"md.engin.umich.edu", 
	"ne.engin.umich.edu", 
	"ri.engin.umich.edu" 
);

@ProcessList = <>;

######################################################################

$version = "spawner v1.0";
$me =      "by Jonathan Mayer, 1994.";
$email =   "jmayer@engin.umich.edu";

($port) = @ARGV;
if (!$port) {
	# pick a session port at random .. is this safe?  nope. ;)
	srand (time^$$);
	$port = 3000+int(rand(1000));
}

print "$version $me  email: $email\n";
print "using port $port for all communications.\n\n";

undef %BusyList;
$launched = 0;
$failed = 0;
$succeeded = 0;

chop ($hostname = `/bin/hostname`);
@xloadprocs = ();
$xloadpos = 0;

sub FORKXLOAD
{
	local ($mach) = @_;
	if (($pid = fork()) == 0) { # CHILD
		setpgrp (0, getppid()); # facilitate later infanticide 
		system (
			$rsh,
			@rshOptions, 
			$mach ,
			" xload " .
			"-display $hostname:0.0 " .
			"-bg maroon -fg yellow " .
			"-geometry 68x68+0-$xloadpos " 
		);
		sleep;
		exit(1);
	} else {
		push (@xloadprocs, $pid);
		$xloadpos += 84;
	}
}

if ($USE_HOSTINFO) {	# don't use built-in host list...
	@MachineList = (); # delete old machine list
	open (HINFO, "$pickmach |");
	while (<HINFO>) {
		chop ($_);
		next if ($_ eq "");
		if (/^(\w+)\./) {
			push (@MachineList, ($1));
###			print STDERR "$1\n";
		}
	}
}

# pop up wall of xloads...
foreach $mach (@MachineList) 
{
	&FORKXLOAD ($mach);
}



sub SIGNAL
{
	local ($msg, $proc, $mach) = @_;

	local($AF_INET) = 2;
	local($SOCK_STREAM) = 1;
	local($sockaddr) = 'S n a4 x8';
	local ($hostname, $them);  chop ($hostname = `hostname`);
	local($name, $aliases, $proto, $len, $thisaddr);
	### local (S);
	($name, $aliases, $proto) = getprotobyname('tcp');
	($name, $aliases, $proto) = getservbyname($port, 'tcp') unless
		port =~ /^\d+$/;;
	($name,$aliases,$type,$len,$thisaddr) =	gethostbyname($hostname);
	($name,$aliases,$type,$len,$thataddr) = gethostbyname($them);

	local($this) = pack($sockaddr, $AF_INET, 0, $thisaddr);
	local($that) = pack($sockaddr, $AF_INET, $port, $thataddr);

	socket (S, $AF_INET, $SOCK_STREAM, $proto) || 
		die "socket: $!\n";
	bind (S, $this) || 
		die "bind: $!\n";
	connect (S, $that) || 
		die "connect: $!\n";
	select (S); $| = 1; select (STDOUT);

	print S "$msg\n$proc\n$mach\n";
	close (S);
}

sub RUN 
{
	local ($pnum);
	local ($proc, $mach) = @_;
	if ($pnum = fork()) {
		# Papa Process
		$BusyList{$mach} = $proc;
		$launched++;
		print STDERR ($mach, ": STARTING ", 
			substr ($proc, -10), ". (proc=$pnum)\n");
	} else {
		# Child Process
		system ("$rsh -n $mach $proc");
		local($v) = $?;
		if ($v) {
			warn ("\$\v = $v\n");
			&SIGNAL ("fail", $proc, $mach);
		} else {
			&SIGNAL ("ok", $proc, $mach);
		}
		exit (0);
	}
	return 0;
}

sub SPAWN
{
	local ($cnt) = 0;
	# while both idle machines and unfinished processes exist...
	while ((@ProcessList) && (@MachineList)) {
		chop($nextProcess = pop (@ProcessList));
		$nextMachine = pop (@MachineList);
		&RUN ($nextProcess, $nextMachine);
		$cnt++;
	}
	return $cnt;
}

# set up sockets...
$AF_INET = 2;
$SOCK_STREAM = 1;

$sockaddr = 'S n a4 x8';

($name, $aliases, $proto) = getprotobyname ('tcp');
if ($port !~ /^\d+$/) {
        ($name, $aliases, $port) = getservbyport ($port, 'tcp');
}

$this = pack($sockaddr, $AF_INET, $port, "\0\0\0\0");

select (NS); $| = 1; select (STDOUT);

socket (S, $AF_INET, $SOCK_STREAM, $proto) || die "socket: $!\n";
bind (S,$this) || die "bind: $!\n";
listen (S,5) || die "connect: $!\n";
select (S); $|=1; select (STDOUT);

while (1) {
	&SPAWN;  # spawn 'em if ya got 'em...

	@busy = keys(%BusyList);
	last if ($#busy < 0);

	($addr = accept (NS,S)) || die "accept: $!";
	chop ($msg = <NS>);
	chop ($proc = <NS>);
	chop ($mach = <NS>);
	# process messages...
	if ($msg eq "ok") {		
		print STDERR ($mach, ": FINISHED ", 
			substr($proc, -10), " :)\n");
		$succeeded++;
		delete $BusyList{$mach};
		push (@MachineList, $mach);
		@busy = keys (%BusyList);
		if ($#busy < 5) {
			###warn ("MACHINES LEFT: ", @busy, "\n");
		}
	}
	elsif ($msg eq "fail") {
		print STDERR ("\n", $mach, ": ******* FAILED ******* ...removing... :( \n\n");
		$failed++;
		delete $BusyList{$mach};
		push (@ProcessList, $proc);
	}
	else {
		warn ("CONFUSED: received unknown signal \"$msg\"!\n");
	}
	close (NS);
}

print "\n\nDone.\n";
print "     processes launched: $launched\n";
print "    processes completed: $succeeded\n";
print "       processes failed: $failed\n";
print "\n";

kill -15, $$;

exit;
### THE END ###
end of spawner ------------------------------------------------ >8

rendgifs ------------------------------------------------------ >8
#! /usr/um/bin/perl

require '/afs/engin.umich.edu/contrib/povray/scripts/CONFIG.INC';

$HELPPAGE = <<HELPPAGE;
 
 rendgifs v.1.0 - renders images as GIFS, and facilitates animation
                  using the ImageMagick animate tool.  Rendgifs uses
                  spawner to distribute raytracing load across multiple
                  machines.  PLEASE USE THIS TOOL RESPONSIBLY.

 syntax: rendgifs <sourcefile> <start> <stop> <incr> <imagedir>

 ex: rendgifs myfile.pov 0 59 1 /tmp/myimages

 public license, Jonathan Mayer, 1994.
 queries: jmayer@engin.umich.edu

HELPPAGE

# ------- configurables ------------
$tracer = $povframe;
$tempdir = "/tmp";
# ----------------------------------

srand (time^$$);
$r = int(rand(9999));
$tempfile = "$tempdir/tmp$r";

if ($#ARGV != 4) {
	print STDERR $HELPPAGE;
	exit(1);
}

print "render v.1.0 - jmayer@engin.umich.edu\n";
($src, $start, $stop, $incr, $imgdir, $mpegfile) = @ARGV;
print "\n";

chop ($hostname = `/bin/hostname`);

print "building process list...\n";

open (PLIST, ">$tempfile") || die "Could open $tempfile: $!\n";

for ($clock = $start; $clock <= $stop; $clock+=$incr) {
	$l = "$tracer $src $clock $hostname $imgdir/frame$clock.gif\n";
	# print $l;
	print PLIST $l;
}
close (PLIST);

system "$spawner < $tempfile";

$ok = 1;
@list = ();
for ($clock = $start; $clock <= $stop; $clock+=$incr) {
	if (-e "$imgdir/frame$clock.gif") {} else {
		print "MISSING: $imgdir/frame$clock.gif\n";
		$ok = 0;
	}
	push (@list, "$imgdir/frame$clock.gif");
}

open (RUNFILE, ">$imgdir/run") || die ("Couldn't create file $imgdir/run");
print RUNFILE "\#! /usr/um/bin/perl\n";
print RUNFILE "\n";
print RUNFILE "\$animate = '$animate';\n";
print RUNFILE "@options = ('-colormap', 'Private');\n";

print RUNFILE '@list = (',"\n";
foreach $f (@list) {
	print RUNFILE "\t\'$f\',\n";
}
print RUNFILE ");\n\n";
print RUNFILE 'exec ($animate, @options, @list);',"\n\n";
close (RUNFILE);
chmod 0755, "$imgdir/run";

exit(0); # don't build MPEG...

end of rendgifs ----------------------------------------------- >8

rendmpeg ------------------------------------------------------ >8
#! /usr/um/bin/perl

require '/afs/engin.umich.edu/contrib/povray/scripts/CONFIG.INC';
$HELPPAGE = <<HELPPAGE;
 
 rendmpeg v.1.0 - renders images and compiles into an MPEG, using
                  POVRAY and mpeg_encode.  Rendmpeg uses spawner to
                  distribute raytracing load across multiple machines.
                  PLEASE USE THIS TOOL RESPONSIBLY.

 syntax: rendmpeg <sourcefile> <start> <stop> <incr> <imagedir> <mpegname>

 public license, Jonathan Mayer, 1994.
 queries: jmayer@engin.umich.edu

HELPPAGE

# ------- configurables ------------
$tracer = $povframe;
$tempdir = "/tmp";
# ----------------------------------

srand (time^$$);
$r = int(rand(9999));
$tempfile = "$tempdir/tmp$r";

if ($#ARGV != 5) {
	print STDERR $HELPPAGE;
	exit(1);
}

print "render v.1.0 - jmayer@engin.umich.edu\n";
($src, $start, $stop, $incr, $imgdir, $mpegfile) = @ARGV;
print "\n";

if (!-e $mpeg_encode) {
	print STDERR "ERROR: mpeg_encode binary not found.\n";
	print STDERR "Jonathan says: \"Maybe it hasn't been compiled\n";
	print STDERR "               for this platform yet?  Try a SUN.\"\n";
	print STDERR "Aborted.\n";
	exit(1);
}

chop ($hostname = `/bin/hostname`);

print "building process list...\n";

open (PLIST, ">$tempfile") || die "Could open $tempfile: $!\n";

for ($clock = $start; $clock <= $stop; $clock+=$incr) {
	$l = "$tracer $src $clock $hostname $imgdir/frame$clock.ppm\n";
	# print $l;
	print PLIST $l;
}
close (PLIST);

system "$spawner < $tempfile";

$ok = 1;
for ($clock = $start; $clock <= $stop; $clock+=$incr) {
	if (-e "$imgdir/frame$clock.ppm") {} else {
		print "MISSING: $imgdir/frame$clock.ppm\n";
		$ok = 0;
	}
}

if (!$ok) {
	print "ERROR: missing frames!  halting...\n";
	exit(1);
}

# write the mpeg_encode parameter file...
open (PARAM, ">$tempfile");
print PARAM "\# mpeg_encode parameter file generated at ", time, "\n";
print PARAM "\n";
print PARAM "PATTERN	IBBPBBPBBPBBPBB\n";
print PARAM "OUTPUT	$mpegfile\n";
print PARAM "BASE_FILE_FORMAT	PPM\n";
print PARAM "INPUT_CONVERT	*\n";
print PARAM "\n";
print PARAM "GOP_SIZE	30\n";
print PARAM "SLICES_PER_FRAME	1\n";
print PARAM "\n";
print PARAM "INPUT_DIR	$imgdir\n";
print PARAM "\n";
print PARAM "INPUT\n";
print PARAM "frame*.ppm		[$start-$stop+$incr]\n";
print PARAM "END_INPUT\n";
print PARAM "\n";
print PARAM "# motion search and qscale params...\n";
print PARAM "PIXEL	HALF\n";
print PARAM "RANGE	10\n";
print PARAM "PSEARCH_ALG	LOGARITHMIC\n";
print PARAM "BSEARCH_ALG	CROSS2\n";
print PARAM "IQSCALE		8\n";
print PARAM "PQSCALE		10\n";
print PARAM "BQSCALE		25\n";
print PARAM "REFERENCE_FRAME ORIGINAL\n";
close (PARAM);

print "\n\nbuilding MPEG:\n\n";

system ("$mpeg_encode $tempfile");

exit(1);

end of rendmpeg ----------------------------------------------- >8

povframe ------------------------------------------------------ >8
#! /usr/um/bin/perl

require '/afs/engin.umich.edu/contrib/povray/scripts/CONFIG.INC';

if ($#ARGV != 3) {
	die ("Incorrect number of parameters, eh?");
}
($src, $clock, $client, $outfile) = @ARGV;

if (!$client) { $client = 'ttl'; }
if (!$outfile) { $outfile = 'frame.gif'; }

$home = $ENV{'HOME'};
$user = $ENV{'USER'};

$tracer = $povray;
@options = (
	"+A",
	"+H240", "+W320",
	#"+H480", "+W640",
	"+FT",
	"+L$POVINC",
	"+L$home",
	"-P",
	"-V",
);
$tempdir = "/tmp";
$tempfile = "$tempdir/temp.$user.tga";

$outputpipe = 	"$tgatoppm | " .
		"$ppmquant 256 2> /dev/null | " .
		"$ppmtogif 2> /dev/null | " .
		"$rsh $client \"cat > $outfile\" 2> /tmp/session.$user ";

open (TRACER, "$tracer +I$src +O$tempfile +K$clock @options 2> /tmp/session.$user |");
$ok = 0;
while (<TRACER>) {
	#print "--> $_";
	if (/Time For Trace:\s+(\d+) hours\s+(\d+) minutes/) { 
		$ok = 1;
		print "$src: frame=$clock, t=$1:$2.\n";
	 }
}
if (!$ok) {
	warn ("tracer failed");
	exit(-1); #ERROR CODE!
}

system ("/bin/cat $tempfile | $outputpipe");
if ($?) {
	die ("output pipe failed");
}
print "Done.\n";
end of povframe ----------------------------------------------- >8

pov ----------------------------------------------------------- >8
#! /usr/um/bin/perl

# A very simple wrapper around POV-Ray by jmayer@engin.umich.edu
# picks some "ideal" command-line options for the user.

require '/afs/engin.umich.edu/contrib/povray/scripts/CONFIG.INC';

($src, $out) = @ARGV;

if (!$out) { $out = "OUT.GIF"; }
if (!$src) { 
	print STDERR "Syntax: POV <src file> [<output file.gif>]\n"; 
	exit(1); 
}

@options = (
	"+A",
	"+H240", "+W320",
	#"+H480", "+W640",
	"+FT",
	"+L$POVINC",
	"+L.",
	"-P",
	"-V",
);
$tempdir = "/tmp";
$tempfile = "$tempdir/temp.tga";

$outputpipe = 	"$tgatoppm | " .
		"$ppmquant 256 2> /dev/null | " .
		"$ppmtogif 2> /dev/null > " .
		"$out";

open (TRACER, "$povray +I$src +O$tempfile @options 2> /tmp/session.log |");
$ok = 0;
while (<TRACER>) {
	#print "--> $_";
	if (/Time For Trace:\s+(\d+) hours\s+(\d+) minutes/) { 
		$ok = 1;
		print "$src: t=$1:$2.\n";
	 }
}
if (!$ok) {
	warn ("tracer failed");
	system ("cat /tmp/session.log");
	exit(1); #ERROR CODE!
}

system ("$cat $tempfile | $outputpipe");
if ($?) {
	die ("output pipe failed");
}
print "Done.\n";
end of pov ---------------------------------------------------- >8



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"On the other hand, the road to hell is paved with melting snowballs." -L.Wall
Jonathan Mayer - jmayer@engin.umich.edu - jmayer@lylahfive.reslife.cornell.edu