Patch: Preliminary Pooma Manual

Jeffrey Oldham oldham at codesourcery.com
Tue Dec 4 01:08:27 UTC 2001


I have been working on a manual for Pooma2.4.  These files will be
part of the manual, mostly written using DocBook.  A portion of the
tutorial introduction chapter and some instructions regarding
downloading and compiling have been written.  The rest is an outline
also containing notes to myself.  Although these files are a work in
progress, I wish to add these files to the Pooma source tree so others
can look at them and so they will be backed up outside of Mountain
View.

2001-Dec-03  Jeffrey D. Oldham  <oldham at codesourcery.com>

        * docs/manual/collateindex.pl: New file copied from DocBook
        distribution.
        * docs/manual/makefile: New file used to create PostScript and
        HTML versions of the manual.
        * docs/manual/outline.xml: New file containing the manual, written
        in DocBook.
        * docs/manual/figures/distributed.mp: New file containing MetaPost
        illustration showing the distributed computing concepts.
        * docs/manual/figures/doof2d.mp: New file containing MetaPost
        illustrations of the Doof2d tutorial implementations.
        * docs/manual/figures/makefile: New file used to create EPS
        figures from the MetaPost files.
        * docs/manual/programs/Doof2d-Array-distributed-annotated.patch:
        New file containing a patch adding callout annotations to
        Doof2d-Array-distributed.cpp.
        * docs/manual/programs/Doof2d-Array-element-annotated.patch:
        Analogous.
        * docs/manual/programs/Doof2d-Array-parallel-annotated.patch:
        Analogous.
        * docs/manual/programs/Doof2d-Array-stencil-annotated.patch:
        Analogous.
        * docs/manual/programs/Doof2d-C-element-annotated.patch:
        Analogous.
        * docs/manual/programs/makefile: New file containing instructions
        to create DocBook versions of programs.
        * examples/Manual/Doof2d/Doof2d-Array-distributed.cpp: New source
        code file containing a simple Doof2d implementation using Arrays
        and distributed computation.
        * examples/Manual/Doof2d/Doof2d-Array-element.cpp: Analogous.
        * examples/Manual/Doof2d/Doof2d-Array-parallel.cpp: Analogous.
        * examples/Manual/Doof2d/Doof2d-Array-stencil.cpp: Analogous.
        * examples/Manual/Doof2d/Doof2d-C-element.cpp: Analogous.
        * examples/Manual/Doof2d/Doof2d-Field-distributed.cpp: Analogous.
        * examples/Manual/Doof2d/Doof2d-Field-parallel.cpp: Analogous.
        * examples/Manual/Doof2d/include.mk: New file needed for
        compilation.
        * examples/Manual/Doof2d/makefile: New file containing instructions
        to compile simple Doof2d implementations.

Thanks,
Jeffrey D. Oldham
oldham at codesourcery.com
-------------- next part --------------
Index: docs/manual/collateindex.pl
===================================================================
RCS file: collateindex.pl
diff -N collateindex.pl
*** /dev/null	Fri Mar 23 21:37:44 2001
--- collateindex.pl	Mon Dec  3 14:01:51 2001
***************
*** 0 ****
--- 1,596 ----
+ # -*- Perl -*-
+ #
+ # $Id: collateindex.pl,v 1.12 2000/01/27 15:07:15 nwalsh Exp $
+ 
+ use Getopt::Std;
+ 
+ $usage = "Usage: $0 <opts> file
+ Where <opts> are:
+        -p        Link to points in the document.  The default is to link
+                  to the closest containing section.
+        -g        Group terms with IndexDiv based on the first letter 
+                  of the term (or its sortas attribute).
+                  (This probably doesn't handle i10n particularly well)
+        -s name   Name the IndexDiv that contains symbols.  The default
+                  is 'Symbols'.  Meaningless if -g is not used.
+        -t name   Title for the index.
+        -P file   Read a preamble from file.  The content of file will
+                  be inserted before the <index> tag.
+        -i id     The ID for the <index> tag.
+        -o file   Output to file. Defaults to stdout.
+        -S scope  Scope of the index, must be 'all', 'local', or 'global'.
+                  If unspecified, 'all' is assumed.
+        -I scope  The implied scope, must be 'all', 'local', or 'global'.
+                  IndexTerms which do not specify a scope will have the
+                  implied scope.  If unspecified, 'all' is assumed.
+        -x        Make a SetIndex.
+        -f        Force the output file to be written, even if it appears
+                  to have been edited by hand.
+        -N        New index (generates an empty index file).
+        file      The file containing index data generated by Jade
+                  with the DocBook HTML Stylesheet.\n";
+ 
+ die $usage if ! getopts('Dfgi:NpP:s:o:S:I:t:x');
+ 
+ $linkpoints   = $opt_p;
+ $lettergroups = $opt_g;
+ $symbolsname  = $opt_s || "Symbols";
+ $title        = $opt_t;
+ $preamble     = $opt_P;
+ $outfile      = $opt_o || '-';
+ $indexid      = $opt_i;
+ $scope        = uc($opt_S) || 'ALL';
+ $impliedscope = uc($opt_I) || 'ALL';
+ $setindex     = $opt_x;
+ $forceoutput  = $opt_f;
+ $newindex     = $opt_N;
+ $debug        = $opt_D;
+ 
+ $indextag     = $setindex ? 'setindex' : 'index';
+ 
+ if ($newindex) {
+     safe_open(*OUT, $outfile);
+     if ($indexid) {
+ 	print OUT "<$indextag id='$indexid'>\n\n";
+     } else {
+ 	print OUT "<$indextag>\n\n";
+     }
+ 
+     print OUT "<!-- This file was produced by collateindex.pl.         -->\n";
+     print OUT "<!-- Remove this comment if you edit this file by hand! -->\n";
+ 
+     print OUT "</$indextag>\n";
+     exit 0;
+ }
+ 
+ $dat = shift @ARGV || die $usage;
+ die "$0: cannot find $dat.\n" if ! -f $dat;
+ 
+ %legal_scopes = ('ALL' => 1, 'LOCAL' => 1, 'GLOBAL' => 1);
+ if ($scope && !$legal_scopes{$scope}) {
+     die "Invalid scope.\n$usage\n";
+ }
+ if ($impliedscope && !$legal_scopes{$impliedscope}) {
+     die "Invalid implied scope.\n$usage\n";
+ }
+ 
+ @term = ();
+ %id   = ();
+ 
+ $termcount = 0;
+ 
+ print STDERR "Processing $dat...\n";
+ 
+ # Read the index file, creating an array of objects.  Each object 
+ # represents and indexterm and has fields for the content of the
+ # indexterm
+ 
+ open (F, $dat);
+ while (<F>) {
+     chop;
+ 
+     if (/^\/indexterm/i) {
+ 	push (@term, $idx);
+ 	next;
+     }
+ 
+     if (/^indexterm (.*)$/i) {
+ 	$termcount++;
+ 	$idx = {};
+ 	$idx->{'zone'} = {};
+ 	$idx->{'href'} = $1;
+ 	$idx->{'count'} = $termcount;
+ 	$idx->{'scope'} = $impliedscope;
+ 	next;
+     }
+ 
+     if (/^indexpoint (.*)$/i) {
+ 	$idx->{'hrefpoint'} = $1;
+ 	next;
+     }
+ 
+     if (/^title (.*)$/i) {
+ 	$idx->{'title'} = $1;
+ 	next;
+     }
+ 
+     if (/^primary[\[ ](.*)$/i) {
+ 	if (/^primary\[(.*?)\] (.*)$/i) {
+ 	    $idx->{'psortas'} = $1;
+ 	    $idx->{'primary'} = $2;
+ 	} else {
+ 	    $idx->{'psortas'} = $1;
+ 	    $idx->{'primary'} = $1;
+ 	}
+ 	next;
+     }
+ 
+     if (/^secondary[\[ ](.*)$/i) {
+ 	if (/^secondary\[(.*?)\] (.*)$/i) {
+ 	    $idx->{'ssortas'} = $1;
+ 	    $idx->{'secondary'} = $2;
+ 	} else {
+ 	    $idx->{'ssortas'} = $1;
+ 	    $idx->{'secondary'} = $1;
+ 	}
+ 	next;
+     }
+ 
+     if (/^tertiary[\[ ](.*)$/i) {
+ 	if (/^tertiary\[(.*?)\] (.*)$/i) {
+ 	    $idx->{'tsortas'} = $1;
+ 	    $idx->{'tertiary'} = $2;
+ 	} else {
+ 	    $idx->{'tsortas'} = $1;
+ 	    $idx->{'tertiary'} = $1;
+ 	}
+ 	next;
+     }
+ 
+     if (/^see (.*)$/i) {
+ 	$idx->{'see'} = $1;
+ 	next;
+     }
+ 
+     if (/^seealso (.*)$/i) {
+ 	$idx->{'seealso'} = $1;
+ 	next;
+     }
+ 
+     if (/^significance (.*)$/i) {
+ 	$idx->{'significance'} = $1;
+ 	next;
+     }
+ 
+     if (/^class (.*)$/i) {
+ 	$idx->{'class'} = $1;
+ 	next;
+     }
+ 
+     if (/^scope (.*)$/i) {
+ 	$idx->{'scope'} = uc($1);
+ 	next;
+     }
+ 
+     if (/^startref (.*)$/i) {
+ 	$idx->{'startref'} = $1;
+ 	next;
+     }
+ 
+     if (/^id (.*)$/i) {
+ 	$idx->{'id'} = $1;
+ 	$id{$1} = $idx;
+ 	next;
+     }
+ 
+     if (/^zone (.*)$/i) {
+ 	my($href) = $1;
+ 	$_ = scalar(<F>);
+ 	chop;
+ 	die "Bad zone: $_\n" if !/^title (.*)$/i;
+ 	$idx->{'zone'}->{$href} = $1;
+ 	next;
+     }
+ 
+     die "Unrecognized: $_\n";
+ }
+ close (F);
+ 
+ print STDERR "$termcount entries loaded...\n";
+ 
+ # Fixup the startrefs...
+ # In DocBook, STARTREF is a #CONREF attribute; support this by copying
+ # all of the fields from the indexterm with the id specified by STARTREF
+ # to the indexterm that has the STARTREF.
+ foreach $idx (@term) {
+     my($ididx, $field);
+     if ($idx->{'startref'}) {
+ 	$ididx = $id{$idx->{'startref'}};
+ 	foreach $field ('primary', 'secondary', 'tertiary', 'see', 'seealso',
+ 		        'psortas', 'ssortas', 'tsortas', 'significance',
+ 		        'class', 'scope') {
+ 	    $idx->{$field} = $ididx->{$field};
+ 	}
+     }
+ }
+ 
+ # Sort the index terms
+ @term = sort termsort @term;
+ 
+ # Move all of the non-alphabetic entries to the front of the index.
+ @term = sortsymbols(@term);
+ 
+ safe_open(*OUT, $outfile);
+ 
+ # Write the index...
+ if ($indexid) {
+     print OUT "<$indextag id='$indexid'>\n\n";
+ } else {
+     print OUT "<$indextag>\n\n";
+ }
+ 
+ print OUT "<!-- This file was produced by collateindex.pl.         -->\n";
+ print OUT "<!-- Remove this comment if you edit this file by hand! -->\n";
+ 
+ print OUT "<!-- ULINK is abused here.
+       
+       The URL attribute holds the URL that points from the index entry
+       back to the appropriate place in the output produced by the HTML
+       stylesheet. (It's much easier to calculate this URL in the first
+       pass.)
+ 
+       The Role attribute holds the ID (either real or manufactured) of
+       the corresponding INDEXTERM.  This is used by the print backends
+       to produce page numbers.
+ 
+       The entries below are sorted and collated into the correct order.
+       Duplicates may be removed in the HTML backend, but in the print
+       backends, it is impossible to suppress duplicate pages or coalesce
+       sequences of pages into a range.
+ -->\n\n";
+ 
+ print OUT "<title>$title</title>\n\n" if $title;
+ 
+ $last = {};     # the last indexterm we processed
+ $first = 1;     # this is the first one
+ $group = "";    # we're not in a group yet
+ $lastout = "";  # we've not put anything out yet
+ 
+ foreach $idx (@term) {
+     next if $idx->{'startref'}; # no way to represent spans...
+     next if ($idx->{'scope'} eq 'LOCAL') && ($scope eq 'GLOBAL');
+     next if ($idx->{'scope'} eq 'GLOBAL') && ($scope eq 'LOCAL');
+     next if &same($idx, $last); # suppress duplicates
+ 
+     $termcount--;
+ 
+     # If primary changes, output a whole new index term, otherwise just
+     # output another secondary or tertiary, as appropriate.  We know from
+     # sorting that the terms will always be in the right order.
+     if (!&tsame($last, $idx, 'primary')) { 
+ 	print "DIFF PRIM\n" if $debug;
+ 	&end_entry() if not $first;
+ 
+ 	if ($lettergroups) {
+ 	    # If we're grouping, make the right indexdivs
+ 	    $letter = $idx->{'psortas'};
+ 	    $letter = $idx->{'primary'} if !$letter;
+ 	    $letter = uc(substr($letter, 0, 1));
+ 	    
+ 	    # symbols are a special case
+ 	    if (($letter lt 'A') || ($letter gt 'Z')) {
+ 		if (($group eq '')
+ 		    || (($group ge 'A') && ($group le 'Z'))) {
+ 		    print OUT "</indexdiv>\n" if !$first;
+ 		    print OUT "<indexdiv><title>$symbolsname</title>\n\n";
+ 		    $group = $letter;
+ 		}
+ 	    } elsif (($group eq '') || ($group ne $letter)) {
+ 		print OUT "</indexdiv>\n" if !$first;
+ 		print OUT "<indexdiv><title>$letter</title>\n\n";
+ 		$group = $letter;
+ 	    }
+ 	}
+ 
+ 	$first = 0; # there can only be on first ;-)
+ 
+ 	print OUT "<indexentry>\n";
+ 	print OUT "  <primaryie>", $idx->{'primary'};
+ 	$lastout = "primaryie";
+ 
+  	if ($idx->{'secondary'}) {
+ 	    print OUT "\n  </primaryie>\n";
+ 	    print OUT "  <secondaryie>", $idx->{'secondary'};
+ 	    $lastout = "secondaryie";
+ 	};
+ 
+ 	if ($idx->{'tertiary'}) {
+ 	    print OUT "\n  </secondaryie>\n";
+ 	    print OUT "  <tertiaryie>", $idx->{'tertiary'};
+ 	    $lastout = "tertiaryie";
+ 	}
+     } elsif (!&tsame($last, $idx, 'secondary')) {
+ 	print "DIFF SEC\n" if $debug;
+ 
+ 	print OUT "\n  </$lastout>\n" if $lastout;
+ 
+ 	print OUT "  <secondaryie>", $idx->{'secondary'};
+ 	$lastout = "secondaryie";
+ 	if ($idx->{'tertiary'}) {
+ 	    print OUT "\n  </secondaryie>\n";
+ 	    print OUT "  <tertiaryie>", $idx->{'tertiary'};
+ 	    $lastout = "tertiaryie";
+ 	}
+     } elsif (!&tsame($last, $idx, 'tertiary')) {
+ 	print "DIFF TERT\n" if $debug;
+ 
+ 	print OUT "\n  </$lastout>\n" if $lastout;
+ 
+ 	if ($idx->{'tertiary'}) {
+ 	    print OUT "  <tertiaryie>", $idx->{'tertiary'};
+ 	    $lastout = "tertiaryie";
+ 	}
+     }
+ 
+     &print_term($idx);
+     
+     $last = $idx;
+ }
+ 
+ # Termcount is > 0 iff some entries were skipped.
+ print STDERR "$termcount entries ignored...\n";
+ 
+ &end_entry();
+ 
+ print OUT "</indexdiv>\n" if $lettergroups;
+ print OUT "</$indextag>\n";
+ 
+ close (OUT);
+ 
+ print STDERR "Done.\n";
+ 
+ sub same {
+     my($a) = shift;
+     my($b) = shift;
+ 
+     my($aP) = $a->{'psortas'} || $a->{'primary'};   
+     my($aS) = $a->{'ssortas'} || $a->{'secondary'}; 
+     my($aT) = $a->{'tsortas'} || $a->{'tertiary'};  
+ 	                                            
+     my($bP) = $b->{'psortas'} || $b->{'primary'};   
+     my($bS) = $b->{'ssortas'} || $b->{'secondary'}; 
+     my($bT) = $b->{'tsortas'} || $b->{'tertiary'};  
+ 
+     my($same);
+ 
+     $aP =~ s/^\s*//; $aP =~ s/\s*$//; $aP = uc($aP);
+     $aS =~ s/^\s*//; $aS =~ s/\s*$//; $aS = uc($aS);
+     $aT =~ s/^\s*//; $aT =~ s/\s*$//; $aT = uc($aT);
+     $bP =~ s/^\s*//; $bP =~ s/\s*$//; $bP = uc($bP);
+     $bS =~ s/^\s*//; $bS =~ s/\s*$//; $bS = uc($bS);
+     $bT =~ s/^\s*//; $bT =~ s/\s*$//; $bT = uc($bT);
+ 
+ #    print "[$aP]=[$bP]\n";
+ #    print "[$aS]=[$bS]\n";
+ #    print "[$aT]=[$bT]\n";
+ 
+     # Two index terms are the same if:
+     # 1. the primary, secondary, and tertiary entries are the same
+     #    (or have the same SORTAS)
+     # AND
+     # 2. They occur in the same titled section
+     # AND
+     # 3. They point to the same place
+     #
+     # Notes: Scope is used to suppress some entries, but can't be used
+     #          for comparing duplicates.
+     #        Interpretation of "the same place" depends on whether or
+     #          not $linkpoints is true.
+ 
+     $same = (($aP eq $bP)
+ 	     && ($aS eq $bS)
+ 	     && ($aT eq $bT)
+ 	     && ($a->{'title'} eq $b->{'title'})
+ 	     && ($a->{'href'} eq $b->{'href'}));
+ 
+     # If we're linking to points, they're only the same if they link
+     # to exactly the same spot.  (surely this is redundant?)
+     $same = $same && ($a->{'hrefpoint'} eq $b->{'hrefpoint'})
+ 	if $linkpoints;
+ 
+     $same;
+ }
+ 
+ sub tsame {
+     # Unlike same(), tsame only compares a single term
+     my($a) = shift;
+     my($b) = shift;
+     my($term) = shift;
+     my($sterm) = substr($term, 0, 1) . "sortas";
+     my($A, $B);
+ 
+     $A = $a->{$sterm} || $a->{$term};
+     $B = $b->{$sterm} || $b->{$term};
+ 
+     $A =~ s/^\s*//; $A =~ s/\s*$//; $A = uc($A);
+     $B =~ s/^\s*//; $B =~ s/\s*$//; $B = uc($B);
+ 
+     return $A eq $B;
+ }
+ 
+ sub end_entry {
+     # End any open elements...
+     print OUT "\n  </$lastout>\n" if $lastout;
+     print OUT "</indexentry>\n\n";
+     $lastout = "";
+ }
+ 
+ sub print_term {
+     # Print out the links for an indexterm.  There can be more than
+     # one if the term has a ZONE that points to more than one place.
+     # (do we do the right thing in that case?)
+     my($idx) = shift;
+     my($key, $indent, @hrefs);
+     my(%href) = ();
+     my(%phref) = ();
+ 
+     $indent = "    ";
+ 
+     if ($idx->{'see'}) {
+ 	# it'd be nice to make this a link...
+ 	if ($lastout) {
+ 	    print OUT "\n  </$lastout>\n";
+ 	    $lastout = "";
+ 	}
+ 	print OUT $indent, "<seeie>", $idx->{'see'}, "</seeie>\n";
+ 	return;
+     }
+ 
+     if ($idx->{'seealso'}) {
+ 	# it'd be nice to make this a link...
+ 	if ($lastout) {
+ 	    print OUT "\n  </$lastout>\n";
+ 	    $lastout = "";
+ 	}
+ 	print OUT $indent, "<seealsoie>", $idx->{'seealso'}, "</seealsoie>\n";
+ 	return;
+     }
+ 
+     if (keys %{$idx->{'zone'}}) {
+ 	foreach $key (keys %{$idx->{'zone'}}) {
+ 	    $href{$key} = $idx->{'zone'}->{$key};
+ 	    $phref{$key} = $idx->{'zone'}->{$key};
+ 	}
+     } else {
+ 	$href{$idx->{'href'}} = $idx->{'title'};
+ 	$phref{$idx->{'href'}} = $idx->{'hrefpoint'};
+     }
+ 
+     # We can't use <LINK> because we don't know the ID of the term in the
+     # original source (and, in fact, it might not have one).
+     print OUT ",\n";
+     @hrefs = keys %href;
+     while (@hrefs) {
+ 	my($linkend) = "";
+ 	my($role) = "";
+ 	$key = shift @hrefs;
+ 	if ($linkpoints) {
+ 	    $linkend = $phref{$key};
+ 	} else {
+ 	    $linkend = $key;
+ 	}
+ 
+ 	$role = $linkend;
+ 	$role = $1 if $role =~ /\#(.*)$/;
+ 
+ 	print OUT $indent;
+ 	print OUT "<ulink url=\"$linkend\" role=\"$role\">";
+ 	print OUT "<emphasis>" if ($idx->{'significance'} eq 'PREFERRED');
+ 	print OUT $href{$key};
+ 	print OUT "</emphasis>" if ($idx->{'significance'} eq 'PREFERRED');
+ 	print OUT "</ulink>";
+     }
+ }
+ 
+ sub termsort {
+     my($aP) = $a->{'psortas'} || $a->{'primary'};   
+     my($aS) = $a->{'ssortas'} || $a->{'secondary'}; 
+     my($aT) = $a->{'tsortas'} || $a->{'tertiary'};  
+     my($ap) = $a->{'count'};
+ 	                                            
+     my($bP) = $b->{'psortas'} || $b->{'primary'};   
+     my($bS) = $b->{'ssortas'} || $b->{'secondary'}; 
+     my($bT) = $b->{'tsortas'} || $b->{'tertiary'};  
+     my($bp) = $b->{'count'};
+ 
+     $aP =~ s/^\s*//; $aP =~ s/\s*$//; $aP = uc($aP);
+     $aS =~ s/^\s*//; $aS =~ s/\s*$//; $aS = uc($aS);
+     $aT =~ s/^\s*//; $aT =~ s/\s*$//; $aT = uc($aT);
+     $bP =~ s/^\s*//; $bP =~ s/\s*$//; $bP = uc($bP);
+     $bS =~ s/^\s*//; $bS =~ s/\s*$//; $bS = uc($bS);
+     $bT =~ s/^\s*//; $bT =~ s/\s*$//; $bT = uc($bT);
+ 
+     if ($aP eq $bP) {
+ 	if ($aS eq $bS) {
+ 	    if ($aT eq $bT) {
+ 		# make sure seealso's always sort to the bottom
+ 		return 1 if ($a->{'seealso'});
+ 		return -1  if ($b->{'seealso'});
+ 		# if everything else is the same, keep these elements
+ 		# in document order (so the index links are in the right
+ 		# order)
+ 		return $ap <=> $bp;
+ 	    } else {
+ 		return $aT cmp $bT;
+ 	    }
+ 	} else {
+ 	    return $aS cmp $bS;
+ 	}
+     } else {
+ 	return $aP cmp $bP;
+     }
+ }
+ 
+ sub sortsymbols {
+     my(@term) = @_;
+     my(@new) = ();
+     my(@sym) = ();
+     my($letter);
+     my($idx);
+ 
+     # Move the non-letter things to the front.  Should digits be thier
+     # own group?  Maybe...
+     foreach $idx (@term) {
+ 	$letter = $idx->{'psortas'};
+ 	$letter = $idx->{'primary'} if !$letter;
+ 	$letter = uc(substr($letter, 0, 1));
+ 
+ 	if (($letter lt 'A') || ($letter gt 'Z')) {
+ 	    push (@sym, $idx);
+ 	} else {
+ 	    push (@new, $idx);
+ 	}
+     }
+ 
+     return (@sym, @new);
+ }
+ 
+ sub safe_open {
+     local(*OUT) = shift;
+     local(*F, $_);
+ 
+     if (($outfile ne '-') && (!$forceoutput)) {
+ 	my($handedit) = 1;
+ 	if (open (OUT, $outfile)) {
+ 	    while (<OUT>) {
+ 		if (/<!-- Remove this comment if you edit this file by hand! -->/){
+ 		    $handedit = 0;
+ 		    last;
+ 		}
+ 	    } 
+ 	    close (OUT);
+ 	} else {
+ 	    $handedit = 0;
+ 	}
+ 	
+ 	if ($handedit) {
+ 	    print "\n$outfile appears to have been edited by hand; use -f or\n";
+ 	    print "      change the output file.\n";
+ 	    exit 1;
+ 	}
+     }
+ 
+     open (OUT, ">$outfile") || die "$usage\nCannot write to $outfile.\n";
+ 
+     if ($preamble) { 
+ 	# Copy the preamble
+ 	if (open(F, $preamble)) {
+ 	    while (<F>) {
+ 		print OUT $_;
+ 	    }
+ 	    close(F);
+ 	} else {
+ 	    warn "$0: cannot open preamble $preamble.\n";
+ 	}
+     }
+ }
Index: docs/manual/makefile
===================================================================
RCS file: makefile
diff -N makefile
*** /dev/null	Fri Mar 23 21:37:44 2001
--- makefile	Mon Dec  3 14:01:51 2001
***************
*** 0 ****
--- 1,69 ----
+ ### Oldham, Jeffrey D.
+ ### 1997 Dec 26
+ ### misc
+ ###
+ ### LaTeX -> PostScript/PDF/WWW
+ ### XML -> TeX/DVI/PS/PDF
+ 
+ # Definitions for PostScript and WWW Creation
+ TEX=		latex
+ WWWHOMEDIR=	/u/oldham/www
+ LATEX2HTML=	latex2html
+ BASICLATEX2HTMLOPTIONS=	-info "" -no_footnode -no_math -html_version 3.2,math
+ #LATEX2HTMLOPTIONS=	-local_icons -split +1 $(BASICLATEX2HTMLOPTIONS)
+ LATEX2HTMLOPTIONS=	-no_navigation -split 0 $(BASICLATEX2HTMLOPTIONS)
+ MPOST=		mpost
+ 
+ # Definitions for Jade.
+ JADEDIR=		/usr/lib/sgml/stylesheets/docbook
+ PRINTDOCBOOKDSL=	print/docbook.dsl
+ HTMLDOCBOOKDSL=		html/docbook.dsl
+ XML=			dtds/decls/xml.dcl
+ INDEXOPTIONS=		-t 'Index' -i 'index' -g -p
+ 
+ CXXFLAGS= -g -Wall -pedantic -W -Wstrict-prototypes -Wpointer-arith -Wbad-function-cast -Wcast-align -Wconversion -Wnested-externs -Wundef -Winline -static
+ 
+ all: outline.ps
+ 
+ %.all:	%.ps %.pdf %.html
+ 	chmod 644 $*.ps $*.pdf
+ 	mv $*.ps $*.pdf $*
+ 
+ %.dvi:	%.ltx
+ 	$(TEX) $<
+ 	# bibtex $*
+ 	# $(TEX) $<
+ 	$(TEX) $<
+ 
+ %.ps:	%.dvi
+ 	dvips -t letter $< -o
+ 
+ %.pdf.ltx:	%.ltx
+ 	sed -e 's/^%\\usepackage{times}/\\usepackage{times}/' $< > $@
+ 
+ %.pdf:	%.pdf.ps
+ 	ps2pdf $< $@
+ 
+ # This rule assumes index creation.
+ %.dvi:	%.xml genindex.sgm
+ 	jade -D$(JADEDIR) -t sgml -d $(HTMLDOCBOOKDSL) -V html-index $(XML) $<
+ 	perl collateindex.pl $(INDEXOPTIONS) -o genindex.sgm HTML.index
+ 	jade -D$(JADEDIR) -t tex -d $(PRINTDOCBOOKDSL) $(XML) $< && jadetex $*.tex && jadetex $*.tex && jadetex $*.tex
+ 
+ genindex.sgm:
+ 	perl collateindex.pl $(INDEXOPTIONS) -N -o $@
+ 
+ %.html:	%.xml
+ 	jade -D$(JADEDIR) -t sgml -d $(HTMLDOCBOOKDSL) $(XML) $<
+ 
+ %.pdf:	%.xml
+ 	jade -D$(JADEDIR) -t tex -d $(PRINTDOCBOOKDSL) $(XML) $< && pdfjadetex $*.tex && pdfjadetex $*.tex
+ 
+ mproof-%.ps:	%.mp
+ 	declare -x TEX=latex && $(MPOST) $< && tex mproof.tex $*.[0-9]* && dvips mproof.dvi -o $@
+ 
+ %.txt:	%.ltx
+ 	detex $< > $@
+ 
+ clean:
+ 	rm -f *.dvi *.aux *.log *.toc *.bak *.blg *.bbl *.glo *.idx *.lof *.lot *.htm *.mpx mpxerr.tex HTML.index outline.tex
Index: docs/manual/outline.xml
===================================================================
RCS file: outline.xml
diff -N outline.xml
*** /dev/null	Fri Mar 23 21:37:44 2001
--- outline.xml	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,4287 ----
+ <?xml version="1.0"?>
+ 
+ <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" "file://usr/lib/sgml/docbookx.dtd" [
+ 
+ <!-- Text to Check, Revise, Replace -->
+   <!-- FIXME: Revise before releasing. -->
+   <!-- UPDATE: Check before publishing to see if any needs changing. -->
+   <!-- ADD: Write more material. -->
+ 
+ <!-- Index Entity Declarations -->
+ <!ENTITY genindex.sgm SYSTEM "genindex.sgm">
+ 
+ <!-- General Entity Declarations -->
+ 
+ <!ENTITY book "book">
+   <!-- Produce a notation for the book/manual/report/WWW page.  -->
+   <!-- Modify this to the desired noun. -->
+ <!ENTITY Book "Book">
+   <!-- Produce a capitalized version of &book;  -->
+   <!-- Modify this to the desired noun. -->
+ <!ENTITY c "<application class='software'>C</application>">
+   <!-- Produce a notation for the C programming language.  -->
+   <!-- Modify this to the desired formatting. -->
+ <!ENTITY cc "<application class='software'>C++</application>">
+   <!-- Produce a notation for the C++ programming language.  -->
+   <!-- Modify this to the desired formatting. -->
+ <!ENTITY cheetah "<application class='software'>Cheetah</application>" >
+   <!-- Produce a notation for the Cheetah Library.  -->
+ <!ENTITY doof2d "<command>Doof2d</command>" >
+   <!-- Produce a notation for the Doof2d program.  -->
+ <!ENTITY make "<application class='software'>Make</application>">
+   <!-- Produce a notation for the GNU Make program.  -->
+ <!ENTITY mm "<application class='software'>MM</application>">
+   <!-- Produce a notation for the MM Library.  -->
+ <!ENTITY mpi "<application class='software'>MPI</application>">
+   <!-- Produce a notation for the MPI package.  -->
+ <!ENTITY pdt "<application class='software'>PDToolkit</application>">
+   <!-- Produce a notation for the PDT software package.  -->
+ <!ENTITY pete "<application class='software'>PETE</application>">
+   <!-- Produce a notation for the PETE library.  -->
+ <!ENTITY pooma "<application class='software'>POOMA</application>">
+   <!-- Produce a notation for Pooma software.  -->
+ <!ENTITY poomaToolkit "<application class='software'>POOMA Toolkit</application>">
+   <!-- Produce a notation for the Pooma toolkit.  -->
+ <!ENTITY purify "<application class='software'>Purify</application>">
+   <!-- Produce a notation for the Purify library.  -->
+ <!ENTITY smarts "<application class='software'>Smarts</application>">
+   <!-- Produce a notation for the Smarts software package.  -->
+   <!-- Modify this after capitalization is decided. -->
+ <!ENTITY stl "<application class='software'>STL</application>">
+   <!-- Produce a notation for the C++ Standard Template Library software package.  -->
+ <!ENTITY tau "<application class='software'>Tau</application>">
+   <!-- Produce a notation for the Tau software package.  -->
+ 
+ <!-- Type Entity Declarations -->
+ 
+ <!ENTITY array "<type>Array</type>">
+   <!-- The "Array" type. -->
+ <!ENTITY benchmark "<type>Benchmark</type>">
+   <!-- The "Benchmark" type. -->
+ <!ENTITY brick "<type>Brick</type>">
+   <!-- The "Brick" engine type. -->
+ <!ENTITY compressiblebrick "<type>CompressibleBrick</type>">
+   <!-- The "CompressibleBrick" engine type. -->
+ <!ENTITY distributedtag "<type>DistributedTag</type>">
+   <!-- The DistributedTag Layout type. -->
+ <!ENTITY domain "<type>Domain</type>">
+   <!-- The "Domain" type. -->
+ <!ENTITY double "<type>double</type>">
+   <!-- The C "double" type. -->
+ <!ENTITY dynamicarray "<type>DynamicArray</type>">
+   <!-- The "DynamicArray" type. -->
+ <!ENTITY engine "<type>Engine</type>">
+   <!-- The "Engine" type. -->
+ <!ENTITY field "<type>Field</type>">
+   <!-- The "Field" type. -->
+ <!ENTITY interval "<type>Interval</type>">
+   <!-- The "Interval" type. -->
+ <!ENTITY layout "<type>Layout</type>">
+   <!-- The "Layout" type without template parameters. -->
+ <!ENTITY leaffunctor "<type>LeafFunctor</type>">
+   <!-- The "LeafFunctor" type. -->
+ <!ENTITY multipatch "<type>MultiPatch</type>">
+   <!-- The "MultiPatch" engine without template parameters. -->
+ <!ENTITY replicatedtag "<type>ReplicatedTag</type>">
+   <!-- The ReplicatedTag Layout type. -->
+ <!ENTITY stencil "<type>Stencil</type>">
+   <!-- The "Stencil" type. -->
+ <!ENTITY vector "<type>Vector</type>">
+   <!-- The "Vector" type. -->
+ 
+ <!-- Mathematical Entity Declarations -->
+ 
+ <!ENTITY n "n">
+   <!-- the size of one dimension of an array -->
+ 
+ <!-- System and Operating System Entity Declarations -->
+ <!ENTITY gcc "<application>g++</application>">
+   <!-- The GNU Compiler Collection C++ compiler. -->
+ <!ENTITY kcc "<application>KCC</application>">
+   <!-- The KAI C++ compiler. -->
+ <!ENTITY linux "<application>Linux</application>">
+   <!-- The Linux operating system. -->
+ 
+ <!-- &pooma; URLs and Files -->
+ 
+ <!ENTITY poomaDownloadPage '<ulink url="http://pooma.codesourcery.com/pooma/download">http://pooma.codesourcery.com/pooma/download</ulink>'>
+   <!-- The WWW page supporting downloading the &pooma; source code. -->
+   <!-- UPDATE this URL. -->
+ <!ENTITY poomaHomePage '<ulink url="http://www.pooma.com/">http://www.pooma.com/</ulink>'>
+   <!-- The canonical Pooma home page. -->
+   <!-- UPDATE this filename. -->
+ <!ENTITY poomaSource "pooma-2.3.0">
+   <!-- The Pooma source code directory. -->
+ <!ENTITY poomaSourceFile "&poomaSource;.tgz">
+   <!-- The Pooma source code archive. -->
+ <!ENTITY poomaExampleDirectory "examples/Manual">
+   <!-- The directory holding this manual's example codes. -->
+ 
+ <!-- Spelling and Formatting Decisions -->
+ <!ENTITY author "author">
+   <!-- A word describing an author xor authors. -->
+   <!-- spelling: element-wise, not elementwise -->
+   <!-- phrase: function object, not functor -->
+   <!-- spelling: multidimensional, not multi-dimensional -->
+   <!-- spelling: multiprocessor, not multi-processor -->
+   <!-- spelling: nonzero, not non-zero -->
+ 
+ <!-- External File Entities -->
+ <!ENTITY doof2d-c-element SYSTEM "./programs/Doof2d-C-element-annotated.cpp">
+   <!-- hand-coded Doof2d implementation -->
+ <!ENTITY doof2d-array-element SYSTEM "./programs/Doof2d-Array-element-annotated.cpp">
+   <!-- Array element-wise Doof2d implementation -->
+ <!ENTITY doof2d-array-parallel SYSTEM "./programs/Doof2d-Array-parallel-annotated.cpp">
+   <!-- Array data-parallel Doof2d implementation -->
+ <!ENTITY doof2d-array-stencil SYSTEM "./programs/Doof2d-Array-stencil-annotated.cpp">
+   <!-- Array stencil Doof2d implementation -->
+ <!ENTITY doof2d-array-distributed SYSTEM "./programs/Doof2d-Array-distributed-annotated.cpp">
+   <!-- distributed Array stencil Doof2d implementation -->
+ ]>
+ 
+ <book>
+  <bookinfo>
+   <title>&pooma;</title>
+   <subtitle>A &cc; Toolkit for High-Performance Parallel Scientific Computing</subtitle>
+   <author><firstname>Jeffrey</firstname><othername
+   role='mi'>D.</othername><surname>Oldham</surname>
+    <affiliation>
+     <orgname>CodeSourcery, LLC</orgname>
+    </affiliation>
+   </author>
+   <copyright><year>2001</year><holder>CodeSourcery, LLC (<ulink url="http://www.codesourcery.com"></ulink>)</holder></copyright>
+   <contractsponsor>Los Alamos National Laboratory<ulink url="http://www.lanl.gov"></ulink></contractsponsor>
+   <legalnotice>
+    <!-- FIXME: What is the correct legal notice? -->
+    <para>All rights reserved.  This document may not be redistributed in any form without the express permission of the author.</para>
+   </legalnotice>
+   <revhistory>
+    <revision>
+     <revnumber>0.01</revnumber>
+     <date>2001 Nov 26</date>
+     <authorinitials>jdo</authorinitials>
+     <revremark>first draft</revremark>
+    </revision>
+   </revhistory>
+  </bookinfo>
+ 
+  <!-- FINISH: May we have a short table of contents followed by a -->
+  <!-- complete table of contents? -->
+ 
+  <preface id="preface">
+   <title>Preface</title>
+ 
+   <para>FINISH: Describe the target audience for &pooma; programs and
+   for this manual: &cc; programmers writing scientific code, possibly
+   parallel execution.</para>
+ 
+   <para>Assume familiarity with &cc; template programming and the
+   standard template library.  FIXME: Remove this index
+   entry.<indexterm id="oldham"><primary>Oldham,
+   Jeffrey D.</primary></indexterm></para>
+ 
+   <section id="preface-notation">
+    <title>Notation</title>
+ 
+    <para>UNFINISHED</para>
+   </section>
+ 
+ 
+   <section id="preface-reading_book:">
+    <title>How to Read This &Book;</title>
+ 
+    <para>FINISH: Write this section in a style similar to Lamport's
+    LaTeX section 1.2.  FINISH: Fix the book title and the section
+    number.</para>
+   </section>
+ 
+ 
+   <section id="preface-downloading">
+    <title>Obtaining &pooma; and Sample Programs</title>
+ 
+    <para>Available for free from what WWW site?  Include what portions
+    of <filename class="libraryfile">LICENSE</filename>?  Be sure to
+    include CVS instructions as well.</para>
+ 
+    <para>Which additional packages are necessary and when?</para>
+ 
+   </section>
+ 
+ 
+   <section id="preface-using_modifying">
+    <title>Using and Modifying &pooma;</title>
+ 
+    <para>&pooma; is available under open source license.  It can be
+    used and modified by anyone, anywhere.  Can it be sold?  Include
+    <filename class="libraryfile">LICENSE</filename>.</para>
+ 
+    <para>QUESTION: How do developers contribute code?</para>
+ 
+   </section>
+ 
+  </preface>
+ 
+ 
+  <part id="programming">
+   <title>Programming with &pooma;</title>
+ 
+   <chapter id="introduction">
+    <title>Introduction</title>
+ 
+    <para>QUESTION: Add a partintro to the part above?</para>
+ 
+    <para>&pooma; abbreviates <quote>Parallel Object-Oriented Methods
+    and Application</quote>.</para>
+ 
+    <para>This document is an introduction to &pooma; v2.1, a &cc;
+    toolkit for high-performance scientific computation.  &pooma;
+    runs efficiently on single-processor desktop machines,
+    shared-memory multiprocessors, and parallel supercomputers
+    containing dozens or hundreds of processors. What's more, by making
+    extensive use of the advanced features of the ANSI/ISO &cc;
+    standard—particularly templates—&pooma; presents a
+    compact, easy-to-read interface to its users.</para>
+ 
+    <para>From Section  of
+    <filename>papers/iscope98.pdf</filename>:</para>
+ 
+    <para>Scientific software developers have struggled with the need
+    to express mathematical abstractions in an elegant and maintainable
+    way without sacrificing performance.  The &pooma; (Parallel
+    Object-Oriented Methods and Applications) framework, written in
+    <acronym>ANSI</acronym>/<acronym>ISO</acronym> &cc;, has
+    demonstrated both high expressiveness and high performance for
+    large-scale scientific applications on platforms ranging from
+    workstations to massively parallel supercomputers.  &pooma; provides
+    high-level abstractions for multidimensional arrays, physical
+    meshes, mathematical fields, and sets of particles.  &pooma; also
+    exploits techniques such as expression templates to optimize serial
+    performance while encapsulating the details of parallel
+    communication and supporting block-based data compression.
+    Consequently, scientists can quickly assemble parallel simulation
+    codes by focusing directly on the physical abstractions relevant to
+    the system under study and not the technical difficulties of
+    parallel communication and machine-specific optimization.</para>
+ 
+    <para>ADD: diagram of science and &pooma;.  See the diagram that
+    Mark and I wrote.</para>
+ 
+ 
+    <section id="introduction-pooma_evolution">
+     <title>Evolution of &pooma;</title>
+ 
+     <para>QUESTION: Is this interesting?  Even if it is, it should be
+     short.</para>
+ 
+     <para>The file <filename>papers/SCPaper-95.html</filename>
+     describes ?&pooma;1? and its abstraction layers.</para>
+ 
+     <para>The "Introduction" of
+     <filename>papers/Siam0098.ps</filename> describes the DoE's
+     funding motivation for &pooma;: Accelerated Strategic Computing
+     Initiative (ASCI) and Science-based Stockpile Stewardship (SBSS),
+     pp. 1–2.</para>
+ 
+     <para>See list of developers on p. 1 of
+     <filename>papers/pooma.ps</filename>.</para>
+ 
+     <para>See list of developers on p. 1 of
+     <filename>papers/pooma.ps</filename>.  See history and motivation
+     on p. 3 of <filename>papers/pooma.ps</filename>.</para>
+ 
+     <para>Use <filename class="libraryfile">README</filename> for
+     information.</para>
+ 
+     <blockquote>
+      <attribution><filename
+ 			    class="libraryfile">introduction.html</filename></attribution>
+ 
+      <para>&pooma; was designed and implemented by scientists working
+      at the Los Alamos National Laboratory's Advanced Computing
+      Laboratory. Between them, these scientists have written and tuned
+      large applications on almost every commercial and experimental
+      supercomputer built in the last two decades. As the technology
+      used in those machines migrates down into departmental computing
+      servers and desktop multiprocessors, &pooma; is a vehicle for its
+      designers' experience to migrate as well. In particular,
+      &pooma;'s authors understand how to get good performance out of
+      modern architectures, with their many processors and multi-level
+      memory hierarchies, and how to handle the subtly complex problems
+      that arise in real-world applications.</para>
+     </blockquote>
+ 
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="tutorial">
+    <title>A Tutorial Introduction</title>
+ 
+    <para>UPDATE: In the following paragraph, fix the cross-reference
+    to the actual section.</para>
+ 
+    <para>&pooma; provides different containers and processor
+    configurations and supports different implementation styles, as
+    described in <xref linkend="introduction"></xref>.  In this
+    chapter, we present several different implementations of the
+    &doof2d; two-dimensional diffusion simulation program:
+     <itemizedlist spacing="compact">
+      <listitem>
+       <para>a C-style implementation omitting any use of &pooma;
+       computing each array element individually,</para>
+      </listitem>
+      <listitem>
+       <para>a &pooma; &array; implementation computing each array
+       element individually,</para>
+      </listitem>
+      <listitem>
+       <para>a &pooma; &array; implementation using data-parallel
+       statements,</para>
+      </listitem>
+      <listitem>
+       <para>a &pooma; &array; implementation using stencils, which
+       support local computations,</para>
+      </listitem>
+      <listitem>
+       <para>a stencil-based &pooma; &array; implementation supporting
+       computation on multiple processors</para>
+      </listitem>
+      <listitem>
+       <para>a &pooma; &field; implementation using data-parallel
+       statements, and</para>
+      </listitem>
+      <listitem>
+       <para>a data-parallel &pooma; &field; implementation for
+       multi-processor execution.</para>
+      </listitem>
+     </itemizedlist>
+    </para>
+    <para>These illustrate the &array;, &field;, &engine;, layout,
+    mesh, and domain data types.  They also illustrate various
+    immediate computation styles (element-wise accesses, data-parallel
+    expressions, and stencil computation) and various processor
+    configurations (one sequential processor and multiple
+    processors).</para>
+ 
+    <figure float="1" id="tutorial-doof2d_averagings">
+     <title>&doof2d; Averagings</title>
+     <mediaobject>
+      <imageobject>
+       <imagedata fileref="figures/doof2d.201" format="EPS" align="center"></imagedata>
+      </imageobject>
+      <textobject>
+       <phrase>The Initial Configuration</phrase>
+      </textobject>
+     </mediaobject>
+     <mediaobject>
+      <imageobject>
+       <imagedata fileref="figures/doof2d.202" format="EPS"></imagedata>
+      </imageobject>
+      <textobject>
+       <phrase>After the First Averaging</phrase>
+      </textobject>
+     </mediaobject>
+     <mediaobject>
+      <imageobject>
+       <imagedata fileref="figures/doof2d.203" format="EPS"></imagedata>
+      </imageobject>
+      <textobject>
+       <phrase>After the Second Averaging</phrase>
+      </textobject>
+     </mediaobject>
+    </figure>
+ 
+    <para>The &doof2d; diffusion program starts with a two-dimensional
+    grid of values.  To model an initial density, all grid values are
+    zero except for one nonzero value in the center.  Each averaging,
+    each grid element, except the outermost ones, updates its value by
+    averaging its value and its eight neighbors.  To avoid overwriting
+    grid values before all their uses occur, we use two arrays, reading
+    the first and writing the second and then reversing their roles
+    within each iteration.</para>
+ 
+    <para>Figure <xref linkend="tutorial-doof2d_averagings"></xref>
+    illustrates the averagings.  Initially, only the center element has
+    nonzero value.  To form the first averaging, each element's new
+    value equals the average of its and its neighbors' previous values.
+    Thus, the initial nonzero value spreads to a three-by-three grid.
+    The averaging continues, spreading to a five-by-five grid of
+    nonzero values.  Values in outermost grid cells are always
+    zero.</para>
+ 
+    <para>Before presenting various implementations of %doof2d;, we
+    explain how to install the &poomaToolkit;.</para>
+ 
+    <para>REMOVE: &doof2d; algorithm and code is illustrated in
+    Section 4.1 of
+    <filename>pooma-publications/pooma.ps</filename>.  It includes a
+    figure illustrating parallel communication of data.</para>
+ 
+    <section id="tutorial-installation">
+     <title>Installing &pooma;</title>
+ 
+     <para>ADD: How does one install &pooma; using Windows or Mac?</para>
+ 
+     <para>UPDATE: Make a more recent &pooma; source code file
+     available on &poomaDownloadPage;.  For example,
+     <quote>LINUXgcc.conf</quote> is not available.</para>
+ 
+     <para>In this section, we describe how to obtain, build, and
+     install the &poomaToolkit;.  We focus on installing under the
+     Unix operating system.  Instructions for installing on computers
+     running Microsoft Windows or MacOS, as well as more extensive
+     instructions for Unix, appear in <xref
+     linkend="installation"></xref>.</para>
+ 
+     <para>Obtain the &pooma; source code <filename
+     path="http://www.codesourcery.com/pooma/downloads_folder/">&poomaSourceFile;</filename>
+     from the &pooma; download page (&poomaDownloadPage;) available off
+     the &pooma; home page (&poomaHomePage;).  The <quote>tgz</quote>
+     indicates this is a compressed tar archive file.  To extract the
+     source files, use <command>tar xzvf &poomaSourceFile;</command>.
+     Move into the source code directory <filename
+     class="directory">&poomaSource;</filename> directory; e.g.,
+     <command>cd &poomaSource;</command>.</para>
+ 
+     <para>Configuring the source code prepares the necessary paths for
+     compilation.  First, determine a configuration file in
+     corresponding to your operating system and compiler in the
+     <filename class="directory">config/arch/</filename> directory.
+     For example, <filename
+     class="libraryfile">LINUXgcc.conf</filename> supports compiling
+     under a &linux; operating system with &gcc; and <filename
+     class="libraryfile">SGI64KCC.conf</filename> supports compiling
+     under a 64-bit <application>SGI</application> Unix operating
+     system with &kcc;.  Then, configure the source code:
+     <command>./configure --arch LINUXgcc --opt --suite
+     LINUXgcc-opt</command>.  The architecture argument to the
+     <command>--arch</command> option is the name of the corresponding
+     configuration file, omitting its <filename
+     class="libraryfile">.conf</filename> suffix.  The
+     <command>--opt</command> indicates the &poomaToolkit; will
+     contain optimized source code, which makes the code run more
+     quickly but may impede debugging.  Alternatively, the
+     <command>--debug</command> option supports debugging.  The
+     <glossterm linkend="glossary-suite_name">suite name</glossterm>
+     can be any arbitrary string.  We chose
+     <command>LINUXgcc-opt</command> to remind us of the architecture
+     and optimization choice.  <filename
+     class="libraryfile">configure</filename> creates subdirectories
+     named by the suite name <quote>LINUXgcc-opt</quote> for use when
+     compiling the source files.  Comments at the beginning of
+     <filename
+     class="libraryfile">lib/<replaceable>suiteName</replaceable>/PoomaConfiguration.h</filename>
+     record the configuration arguments.</para>
+ 
+     <para>To compile the source code, set the
+     <envar>POOMASUITE</envar> environment variable to the suite name
+     and then type <command>make</command>.  To set the environment
+     variable for the <application>bash</application> shell use
+     <command>export
+     POOMASUITE=<replaceable>suiteName</replaceable></command>,
+     substituting the suite name's
+     <replaceable>suiteName</replaceable>.  For the
+     <application>csh</application> shell, use <command>setenv
+     POOMASUITE LINUXgcc-opt</command>.  Issuing the
+     <command>make</command> command compiles the &pooma; source code
+     files to create the &pooma; library.  The &pooma; makefiles assume
+     the <trademark>GNU</trademark> &make; so substitute the proper
+     command if necessary.  The &pooma; library can be found in, e.g.,
+     <filename
+     class="libraryfile">lib/LINUXgcc-opt/libpooma-gcc.a</filename>.</para>
+    </section>
+ 
+    <section id="tutorial-hand_coded">
+     <title>Hand-Coded Implementation</title>
+ 
+     <para>Before implementing &doof2d; using the &poomaToolkit;, we
+     present a hand-coded implementation of &doof2d;.  See <xref
+     linkend="tutorial-hand_coded-doof2d"></xref>.  After querying the
+     user for the number of averagings, the arrays' memory is
+     allocated.  Since the arrays' size is not known at compile time,
+     the arrays are accesses via pointers to allocated dynamic memory.
+     This memory is deallocated at the program's end to avoid memory
+     leaks.  The arrays are initialized with initial conditions.  For
+     the <varname>b</varname> array, all values except the central ones
+     have nonzero values.  Only the outermost values of the
+     <varname>a</varname> array need be initialized to zero, but we
+     instead initialize them all using the loop used by
+     <varname>b</varname>.</para>
+ 
+     <para>The simulation's kernel consists of triply nested loops.
+     The outermost loop controls the number of iterations.  The inner
+     nested loops iterate through the arrays' elements, excepting the
+     outermost elements; note the loop indices range from 1 to n-2
+     while the array indices range from 0 to n-1.  Each
+     <varname>a</varname> value is assigned the average of its
+     corresponding value in <varname>b</varname> and the latter's
+     neighbors.  Values in the two-dimensional grids are accessed using
+     two sets of brackets, e.g., <statement>a[i][j]</statement>.  After
+     assigning values to <varname>a</varname>, a second averaging reads
+     values in <varname>a</varname>, writing values in
+     <varname>b</varname>.</para>
+ 
+     <para>After the kernel finishes, the final central value is
+     printed.  If the desired number of averagings is even, the value
+     in <varname>b</varname> is printed; otherwise, the value in
+     <varname>a</varname> is used.  Finally, the dynamically-allocated
+     memory must be freed to avoid memory leaks.</para>
+ 
+     <example id="tutorial-hand_coded-doof2d">
+      <title>Hand-Coded Implementation of &doof2d;</title>
+      &doof2d-c-element;
+      <calloutlist>
+       <callout arearefs="tutorial-hand_coded-doof2d-nuaveragings">
+        <para>The user specifies the desired number of averagings.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-array_storage">
+        <para>These variables point to the two-dimensional,
+        dynamically-allocated grids so we use a pointer to a pointer to
+        a &double;.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-grid_size">
+        <para>The user enters the desired grid size.  The grid will be
+        a square with <varname>n</varname> by <varname
+        lang="c++">n</varname> grid cells.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-allocation">
+        <para>Memory for the arrays is allocated.  By default, the
+        array indices are zero-based.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-initialization">
+        <para>Initially, all grid values are zero except for the one
+        nonzero value at the center of the second array.  Array
+        positions are indicated using two brackets, e.g.,
+        <statement>a[i][j]</statement>.  A better implementation might
+        initialize only the outermost values of the
+        <varname>a</varname> array.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-constants">
+        <para>These constants indicate the number of iterations, and
+        the average weighting.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-first_write">
+        <para>Each <varname>a</varname> value, except an outermost one,
+        is assigned the average of its analogous <varname>b</varname>
+        value and that value's neighbors.  Note the loop indices ensure
+        the outermost values are not changed.  The
+        <varname>weight</varname>'s value ensures the computation is an
+        average.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-second_write">
+        <para>The second averaging computes <varname>b</varname>'s
+        values using values stored in <varname>a</varname>.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-answer">
+        <para>After the averagings finish, the central value is printed.</para>
+       </callout>
+       <callout arearefs="tutorial-hand_coded-doof2d-deallocation">
+        <para>The dynamically-allocated memory must be deallocated to
+        avoid memory leaks.</para>
+       </callout>
+      </calloutlist>
+     </example>
+ 
+     <para>To compile the executable, change directories to the &pooma;
+     <filename
+     class="directory">&poomaExampleDirectory;/Doof2d</filename>
+     directory.  Ensure the <envar>POOMASUITE</envar> environment
+     variable specifies the desired suite name
+     <replaceable>suiteName</replaceable>, as we did when compiling
+     &pooma; in the previous section <xref
+     linkend="tutorial-installation"></xref>.  Issuing the
+     <command>make Doof2d-C-element</command> command creates the
+     executable
+     <command><replaceable>suiteName</replaceable>/Doof2d-C-element</command>.</para>
+ 
+     <para>When running the executable, specify the desired a
+     nonnegative number of averagings and the nonnegative number of
+     grid cells along any dimension.  The resulting grid has the same
+     number of cells along each dimension.  After the executable
+     finishes, the resulting value of the central element is
+     printed.</para>
+    </section>
+ 
+ 
+    <section id="tutorial-array_elementwise">
+     <title>Element-wise &array; Implementation</title>
+ 
+     <para>The simplest way to use the &poomaToolkit; is to
+     use the &pooma; &array; class instead of &c; arrays.  &array;s
+     automatically handle memory allocation and deallocation, support a
+     wider variety of assignments, and can be used in expressions.
+     <xref linkend="tutorial-array_elementwise-doof2d"></xref>
+     implements &doof2d; using &array;s and element-wise accesses.
+     Since the same algorithm is used as <xref
+     linkend="tutorial-hand_coded-doof2d"></xref>, we will concentrate
+     on the differences.</para>
+ 
+     <example id="tutorial-array_elementwise-doof2d">
+      <title>Element-wise &array; Implementation of &doof2d;</title>
+      &doof2d-array-element;
+      <calloutlist>
+       <callout arearefs="tutorial-array_elementwise-doof2d-header">
+        <para>To use &pooma; &array;s, the <filename
+        class="headerfile">Pooma/Arrays.h</filename> must be included.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-pooma_initialize">
+        <para>The &poomaToolkit; structures must be constructed before
+        their use.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-domain">
+        <para>Before creating an &array;, its domain must be specified.
+        The <varname>N</varname> interval represents the
+        one-dimensional integral set {0, 1, 2, …, n-1}.  An
+        <type>Interval<2></type> object represents the entire
+        two-dimensional index domain.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-array_creation">
+        <para>An &array;'s template parameters indicate its dimension,
+        its value type, and how the values will be stored or computed.
+        The &brick; &engine; type indicates values will be directly
+        stored.  It is responsible for allocating and deallocating
+        storage so <statement>new</statement> and
+        <statement>delete</statement> statements are not necessary.
+        The <varname>vertDomain</varname> specifies the array index
+        domain.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-initialization">
+        <para>The first statement initializes all &array; values to the
+        same scalar value.  This is possible because each &array;
+        <quote>knows</quote> its domain.  The second statement
+        illustrates &array; element access.  Indices, separated by
+        commas, are surrounded by parentheses rather than surrounded by
+        square brackets (<statement>[]</statement>).</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-first_write">
+        <para>&array; element access uses parentheses, rather than
+        square brackets</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-blockAndEvaluate">
+        <para>&pooma; may reorder computation of statements.  Calling
+        <function>Pooma::blockAndEvaluate</function> ensures all
+        computation finishes before accessing a particular array
+        element.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-deallocation">
+        <para>Since &array;s are first-class objects, they
+        automatically deallocate any memory they require, eliminating
+        memory leaks.</para>
+       </callout>
+       <callout arearefs="tutorial-array_elementwise-doof2d-pooma_finish">
+        <para>The &poomaToolkit; structures must be destructed after
+        their use.</para>
+       </callout>
+      </calloutlist>
+     </example>
+ 
+     <para>We describe the use of &array; and the &poomaToolkit; in
+     <xref linkend="tutorial-array_elementwise-doof2d"></xref>.
+     &array;s, declared in the <filename
+     class="headerfile">Pooma/Arrays.h</filename>, are first-class
+     objects.  They <quote>know</quote> their index domain, can be used
+     in expressions, can be assigned scalar and array values, and
+     handle their own memory allocation and deallocation.</para>
+ 
+     <para>The creation of the <varname>a</varname> and
+     <varname>b</varname> &array;s requires an object specifying their
+     index domains.  Since these are two-dimensional arrays, their
+     index domains are also two dimensional.  The two-dimensional
+     <type>Interval<2></type> object is the Cartesian product of
+     two one-dimensional <type>Interval<1></type> objects, each
+     specifying the integral set {0, 1, 2, …, n-1}.</para>
+ 
+     <para>An &array;'s template parameters indicate its dimension, the
+     type of its values, and how the values are stored.  Both
+     <varname>a</varname> and <varname>b</varname> are two-dimension
+     arrays storing &double;s so their <varname>dimension</varname>
+     is 2 and its element type is &double;.  An &engine; stores an
+     &array;'s values.  For example, a &brick; &engine; explicitly
+     stores all values.  A &compressiblebrick; &engine; also explicitly
+     stores values if more than value is present, but, if all values
+     are the same, storage for just that value is required.  Since an
+     engine can store its values any way it desires, it might instead
+     compute its values using a function or compute the values stored
+     in separate engines.  In practice, most explicitly specified
+     &engine;s are either &brick; or &compressiblebrick;.</para>
+ 
+     <para>&array;s support both element-wise access and scalar
+     assignment.  Element-wise access uses parentheses, not square
+     brackets.  For example, <statement>b(n/2,n/2)</statement>
+     specifies the central element.  The scalar assignment <statement>b
+     = 0.0</statement> assigns the same 0.0 value to all array
+     elements.  This is possible because the array knows the extent of
+     its domain.</para>
+ 
+     <para>After the kernel finishes, the central value is printed out.
+     Just prior to this &array; access, a call to
+     <statement>Pooma::blockAndEvaluate()</statement> ensures all
+     computation has finished.  &pooma; may reorder computation or
+     distribute them among various processors.  Before reading an
+     individual &array; value, <function>blockAndEvaluate</function>
+     ensures the value has the correct value.  Calling this function is
+     necessary only when accessing individual array elements because
+     &pooma; cannot determine when to call the function itself. For
+     example, before printing an array, &pooma; will call
+     <function>blockAndEvaluate</function> itself.</para>
+ 
+     <para>Any program using the &poomaToolkit; must initialize the
+     toolkit's data structures using
+     <statement>Pooma::initialize(argc,argv)</statement>.  This
+     extracts &pooma;-specific command-line options from the
+     command-line arguments in <varname>argv</varname> and initializes
+     the inter-processor communication and other data structures.  When
+     finished, <statement>Pooma::finalize()</statement> ensures all
+     computation has finished and the communication and other data
+     structures are destructed.</para>
+    </section>
+ 
+ 
+    <section id="tutorial-data_parallel">
+     <title>Data-Parallel &array; Implementation</title>
+ 
+     <para>&pooma; supports data-parallel &array; accesses.  Many
+     algorithms are more easily expressed using data-parallel
+     expressions.  Also, the &poomaToolkit; might be able to reorder
+     the data-parallel computations to be more efficient or distribute
+     them among various processors.  In this section, we concentrate
+     the differences between the data-parallel implementation of
+     &doof2d; listed in <xref
+     linkend="tutorial-array_parallel-doof2d"></xref> and the
+     element-wise implementation listed in the previous section <xref
+     linkend="tutorial-array_elementwise"></xref>.</para>
+ 
+     <example id="tutorial-array_parallel-doof2d">
+      <title>Data-Parallel &array; Implementation of &doof2d;</title>
+      &doof2d-array-parallel;
+      <calloutlist>
+       <callout arearefs="tutorial-array_parallel-doof2d-innerdomain">
+        <para>These variables specify one-dimensional domains {1, 2,
+        …, n-2}.  Their Cartesian product specifies the domain
+        of the array values that are modified.</para>
+       </callout>
+       <callout arearefs="tutorial-array_parallel-doof2d-first_write">
+        <para>Data-parallel expressions replace nested loops and array
+        element accesses.  For example, <statement>a(I,J)</statement>
+        represents the subset of the <varname>a</varname> array having
+        a domain equal to the Cartesian product of <varname>I</varname>
+        and <varname>J</varname>.  Intervals can shifted by an additive
+        or multiplicative constant.</para>
+       </callout>
+      </calloutlist>
+     </example>
+ 
+     <para>Data-parallel expressions apply domain objects to containers
+     to indicate a set of parallel expressions.  For example, in the
+     program listed above, <statement>a(I,J)</statement> specifies all
+     of <varname>a</varname> array excepting the outermost elements.
+     The array's <varname>vertDomain</varname> domain consists of the
+     Cartesian product of {0, 1, 2, …, n-1} and itself, while
+     <varname>I</varname> and <varname>J</varname> each specify {1, 2,
+     …, n-2}.  Thus, <statement>a(I,J)</statement> is the subset
+     with a domain of the Cartesian product of {1, 2, …, n-2}
+     and itself.  It is called a <firstterm>view</firstterm> of an
+     array.  It is itself an array, with a domain and supporting
+     element access, but its storage is the same as
+     <varname>a</varname>'s.  Changing a value in
+     <statement>a(I,J)</statement> also changes the same value in
+     <varname>a</varname>.  Changing a value in the latter also changes
+     the former if the value is not one of <varname>a</varname>'s
+     outermost elements.  The expression
+     <statement>b(I+1,J+1)</statement> indicates the subset of
+     <varname>b</varname> with a domain consisting of the Cartesian
+     product of {2, 3, …, n-1}, i.e., the same domain as
+     <statement>a(I,J)</statement> but shifted up one unit and to the
+     right one unit.  Only an &interval;'s value, not its name, is
+     important.  Thus, all uses of <varname>J</varname> in this program
+     could be replaced by <varname>I</varname> without changing the
+     semantics.</para>
+ 
+     <figure float="1" id="tutorial-array_parallel-doof2d-adding_arrays">
+      <title>Adding &array;s</title>
+      <mediaobject>
+       <imageobject>
+        <imagedata fileref="figures/doof2d.210" format="EPS" align="center"></imagedata>
+       </imageobject>
+       <textobject>
+        <phrase>Adding two arrays with different domains.</phrase>
+       </textobject>
+       <caption>
+        <para>When adding arrays, values in corresponding positions are
+        added even if they have different indices, indicated by the
+        small numbers adjacent to the arrays.</para>
+       </caption>
+      </mediaobject>
+     </figure>
+ 
+     <para>The statement assigning to <statement>a(I,J)</statement>
+     illustrates that &array;s may participate in expressions.  Each
+     addend is a view of an array, which is itself an array.  Each view
+     has the same domain size so their sum can be formed by
+     corresponding elements of each array.  For example, the lower,
+     left element of the result equals the sum of the lower, left
+     elements of the addend arrays.  For the computation, indices are
+     ignored; only the relative positions within each domain are used.
+     <xref
+     linkend="tutorial-array_parallel-doof2d-adding_arrays"></xref>
+     illustrates adding two arrays with different domain indices.  The
+     indices are indicated by the small numbers to the left and the
+     bottom of the arrays.  Even though 9 and 3 have different indices
+     (1,1) and (2,0), they are added to each other because they have
+     the same relative positions within the addends.</para>
+    </section>
+ 
+ 
+    <section id="tutorial-stencil">
+     <title>Stencil &array; Implementation</title>
+ 
+     <para>Many computations are local, computing a &array;'s value by
+     using close-by &array; values.  Encapsulating this computation in
+     a stencil can yield faster code because the compiler can determine
+     all accesses come from the same array.  Each stencil consists of a
+     function object and an indication of the stencil's extent.</para>
+ 
+     <example id="tutorial-array_stencil-doof2d">
+      <title>Stencil &array; Implementation of &doof2d;</title>
+      &doof2d-array-stencil;
+      <calloutlist>
+       <callout arearefs="tutorial-array_stencil-doof2d-stencil">
+        <para>A stencil is a function object implementing a local
+        operation on an &array;.</para>
+       </callout>
+       <callout
+        arearefs="tutorial-array_stencil-doof2d-stencil_operator">
+        <para>&pooma; applies this function call
+        <function>operator()</function> to the interior domain of an
+        &array;.  Although not strictly necessary, the function's
+        template parameter <varname>C</varname> permits using this
+        stencil with &array;s and other containers.  The
+        <function>read</function> &array; member function supports only
+        reading values, not writing values, thus possibly permitting
+        faster access.</para>
+       </callout>
+       <callout arearefs="tutorial-array_stencil-doof2d-stencil_extent">
+        <para>These two functions indicate the stencil's size.  For
+        each dimension, the stencil extends one cell to the left of (or
+        below) its center and also one call to the right (or above) its
+        center.</para>
+       </callout>
+       <callout
+        arearefs="tutorial-array_stencil-doof2d-stencil_creation">
+        <para>Create the stencil.</para>
+       </callout>
+       <callout arearefs="tutorial-array_stencil-doof2d-first_write">
+        <para>Applying <varname>stencil</varname> to the
+        <varname>b</varname> array and a subset
+        <varname>interiorDomain</varname> of its domain yields an
+        array, which is assigned to a subset of <varname>a</varname>.
+        The stencil's function object is applied to each position in
+        the specified subset of <varname>b</varname>.</para>
+       </callout>
+      </calloutlist>
+     </example>
+ 
+     <para>Before we describe how to create a stencil, we describe how
+     to apply a stencil to an array, yielding values.  To compute the
+     value associated with index position (1,3), the stencil's center
+     is placed at (1,3).  The stencil's
+     <function>upperExtent</function> and
+     <function>lowerExtent</function> functions indicate which &array;
+     values the stencil's function will use.  See <xref
+     linkend="tutorial-array_stencil-doof2d-apply_stencil"></xref>.
+     Applying the stencil's function call
+     <function>operator()</function> yields the computed value.  To
+     compute multiple &array; values, apply a stencil to the array and
+     a domain object: <statement>stencil(b,
+     interiorDomain)</statement>.  This applies the stencil to each
+     position in the domain.  The user must ensure that applying the
+     stencil does not access nonexistent &array; values.</para>
+ 
+     <figure float="1" id="tutorial-array_stencil-doof2d-apply_stencil">
+      <title>Applying a Stencil to an &array;</title>
+      <mediaobject>
+       <imageobject>
+        <imagedata fileref="figures/doof2d.211" format="EPS" align="center"></imagedata>
+       </imageobject>
+       <textobject>
+        <phrase>Apply a stencil to position (1,3) of an array.</phrase>
+       </textobject>
+       <caption>
+        <para>To compute the value associated with index position (1,3)
+        of an array, place the stencil's center, indicated with dashed
+        lines, at the position.  The computation involves the array
+        values covered by the array and delineated by
+        <function>upperExtent</function> and
+        <function>lowerExtent</function>.</para>
+       </caption>
+      </mediaobject>
+     </figure>
+ 
+     <para>To create a stencil object, apply the &stencil; type to a
+     function object class.  For example,
+     <statement>Stencil<DoofNinePt> stencil</statement> declares
+     the <varname>stencil</varname> object.  The function object class
+     must define a function call <function>operator()</function> with a
+     container parameter and index parameters.  The number of index
+     parameters, indicating the stencil's center, must equal the
+     container's dimension.  For example, <type>DoofNinePt</type>
+     defines <methodname>operator()(const C& c, int i, int
+     j)</methodname>.  We templated the container type
+     <varname>C</varname> although this is not strictly necessary.  The
+     two index parameters <varname>i</varname> and <varname>j</varname>
+     ensure the stencil works with two-dimensional containers.  The
+     <methodname>lowerExtent</methodname> indicates how far to the left
+     (or below) the stencil extends beyond its center.  Its parameter
+     indicates a particular dimension.  Index parameters
+     <varname>i</varname> and <varname>j</varname> are in dimension 0
+     and 1.  <methodname>upperExtent</methodname> serves an
+     analogous purpose.  The &poomaToolkit; uses these functions when
+     distribution computation among various processors, but it does not
+     use these functions to ensure nonexistent &array; values are not
+     accessed.  Caveat stencil user!</para>
+    </section>
+ 
+ 
+    <section id="tutorial-distributed">
+     <title>Distributed &array; Implementation</title>
+ 
+     <para>A &pooma; program can execute on one or multiple processors.
+     To convert a program designed for uniprocessor execution to a
+     program designed for multiprocessor execution, the programmer need
+     only specify how each container's domain should be split into
+     <quote>patches</quote>.  The &poomaToolkit; automatically
+     distributes the data among the available processors and handles
+     any required communication between processors.</para>
+ 
+     <example id="tutorial-array_distributed-doof2d">
+      <title>Distributed Stencil &array; Implementation of &doof2d;</title>
+      &doof2d-array-distributed;
+      <calloutlist>
+       <callout arearefs="tutorial-array_distributed-doof2d-nuprocessors">
+        <para>The number of processors executing a &pooma; program can
+        be specified at run-time.</para>
+       </callout>
+       <callout arearefs="tutorial-array_distributed-doof2d-layout">
+        <para>The <type>UniformGridPartition</type> declaration
+        specifies how an array's domain will be partition, of split,
+        into patches.  Guard layers are an optimization that can reduce
+        data communication between patches.  The
+        <type>UniformGridLayout</type> declaration applies the
+        partition to the given domain, distributing the resulting
+        patches among various processors.</para>
+       </callout>
+       <callout arearefs="tutorial-array_distributed-doof2d-remote">
+        <para>The <type>MultiPatch</type> &engine; distributes requests
+        for &array; values to the associated patch.  Since a patch may
+        associated with a different processor, its
+        <quote>remote</quote> engine has type
+        <type>Remote<Brick></type>.  &pooma; automatically
+        distributes the patches among available memories and
+        processors.</para>
+       </callout>
+       <callout
+        arearefs="tutorial-array_distributed-doof2d-first_write">
+        <para>The stencil computation, whether for one processor or
+        multiple processors, is the same.</para>
+       </callout>
+      </calloutlist>
+     </example>
+ 
+     <para>Supporting distributed computation requires only minor code
+     changes.  These changes specify how each container's domain is
+     distributed among the available processors.  The rest of the
+     program, including all the computations, remains the same.  When
+     running, the &pooma; executable interacts with the run-time
+     library to determine which processors are available, distributes
+     the containers' domains, and automatically handles all necessary
+     interprocessor communication.  The same executable runs on one or
+     many processors.  Thus, the programmer can write one program,
+     debugging it on a uniprocessor computer and running it on a
+     supercomputer.</para>
+ 
+     <figure float="1" id="tutorial-array_distributed-doof2d-distributed_model">
+      <title>The &pooma; Distributed Computation Model</title>
+      <mediaobject>
+       <imageobject>
+        <imagedata fileref="figures/distributed.101" format="EPS" align="center"></imagedata>
+       </imageobject>
+       <textobject>
+        <phrase>the &pooma; distributed computation model.</phrase>
+       </textobject>
+       <caption>
+        <para>The &pooma; distributed computation model combines
+        partitioning containers' domains and the computer configuration
+        to create a layout.</para>
+       </caption>
+      </mediaobject>
+     </figure>
+ 
+     <para>&pooma;'s distributed computing model separates container
+     domain concepts from computer configuration concepts.  See <xref
+     linkend="tutorial-array_distributed-doof2d-distributed_model"></xref>.
+     The program indicates how each container's domain will be
+     partitioned.  This process is represented in the upper left corner
+     of the figure.  A user-specified partition specifies how to split
+     the domain into pieces.  For example, the illustrated partition
+     splits the domain into three equal-sized pieces along the
+     x-dimension and two equal-sized pieces along the y-dimension.
+     Thus, the domain is split into <firstterm>patches</firstterm>.
+     The partition also specifies external and internal guard layers.
+     A <firstterm>guard layer</firstterm> is a domain surrounding a
+     patch.  A patch's computation only reads but does not write these
+     values.  An <firstterm>external guard layer</firstterm>
+     conceptually surrounds the entire container domain with boundary
+     values whose presence permits all domain computations to be
+     performed the same way even for values along the domain's edge.
+     An <firstterm>internal guard layer</firstterm> duplicates values
+     from adjacent patches so communication need not occur during a
+     patch's computation.  The use of guard layers is an optimization;
+     using external guard layers eases programming and using internal
+     guard layers reduces communication between processor.  Their use
+     is not required.</para>
+ 
+     <para>The computer configuration of shared memory and processors
+     is determined by the run-time system.  See the upper right portion
+     of <xref
+     linkend="tutorial-array_distributed-doof2d-distributed_model"></xref>.
+     A <firstterm>context</firstterm> is a collection of shared memory
+     and processors that can execute a program or a portion of a
+     program.  For example, a two-processor desktop computer might have
+     memory accessible to both processors so it is a context.  A
+     supercomputer consisting of desktop computers networked together
+     might have as many contexts as computers.  The run-time system,
+     e.g., the Message Passing Interface (&mpi;) Communications Library
+     (FIXME: xref linkend="mpi99", <ulink
+     url="http://www-unix.mcs.anl.gov/mpi/"></ulink>) or the &mm;
+     Shared Memory Library (<ulink
+     url="http://www.engelschall.com/sw/mm/"></ulink>), communicates
+     the available contexts to the executable.  &pooma; must be
+     configured for the particular run-time system.  See <xref
+     linkend="installation-distributed_computing"></xref>.</para>
+ 
+     <para>A <firstterm>layout</firstterm> combines patches with
+     contexts so the program can be executed.  If &distributedtag; is
+     specified, the patches are distributed among the available
+     contexts.  If &replicatedtag; is specified, each set of patches is
+     replicated among each context.  Regardless, the containers'
+     domains are now distributed among the contexts so the program can
+     run.  When a patch needs data from another patch, the &pooma;
+     toolkit sends messages to the desired patch uses a message-passing
+     library.  All such communication is automatically performed by the
+     toolkit with no need for programmer or user input.</para>
+ 
+     <para>FIXME: The two previous paragraphs demonstrate confusion
+     between <quote>run-time system</quote> and <quote>message-passing
+     library</quote>.</para>
+ 
+     <para>Incorporating &pooma;'s distributed computation model into a
+     program requires writing very few lines of code.  <xref
+     linkend="tutorial-array_distributed-doof2d"></xref> illustrates
+     this.  The <varname>partition</varname> declaration creates a
+     <type>UniformGridPartition</type> splitting each dimension of a
+     container's domain into equally-sized
+     <varname>nuProcessors</varname> pieces.  The first
+     <type>GuardLayers</type> argument specifies each patch will have
+     copy of adjacent patches' outermost values.  This may speed
+     computation because a patch need not synchronize its computation
+     with other patches' processors.  Since each value's computation
+     requires knowing its surrounding neighbors, the internal guard
+     layer is one layer deep.  The second <type>GuardLayers</type>
+     argument specifies no external guard layer.  External guard layers
+     simplify computing values along the edges of domains.  Since the
+     program already uses only the interior domain for computation, we
+     do not use this feature.</para>
+ 
+     <para>The <varname>layout</varname> declaration creates a
+     <type>UniformGridLayout</type> layout.  As <xref
+     linkend="tutorial-array_distributed-doof2d"></xref> illustrates,
+     it needs to know a container's domain, a partition, the computer's
+     contexts, and a &distributedtag; or &replicatedtag;.  These
+     comprise <varname>layout</varname>'s three parameters; the
+     contexts are implicitly supplied by the run-time system.</para>
+ 
+     <para>To create a distributed &array;, it should be created using
+     a &layout; object and have a &multipatch; engine.  Prior
+     implementations designed for uniprocessors constructed the
+     container using a &domain; object.  A distributed implementation
+     uses a &layout; object, which conceptually specifies a &domain;
+     object and its distribution throughout the computer.  A
+     &multipatch; engine supports computations using multiple patches.
+     The <type>UniformTag</type> indicates the patches all have the
+     same size.  Since patches may reside on different contexts, the
+     second template parameter is <type>Remote</type>.  Its
+     <type>Brick</type> template parameter specifies the engine for a
+     particular patch on a particular context.  Most distributed
+     programs use <type>MultiPatch<UniformTag, Remote<Brick>
+     ></type> or <type>MultiPatch<UniformTag,
+     Remote<CompressibleBrick> ></type> engines.</para>
+ 
+     <para>The computations for a distributed implementation are
+     exactly the same as for a sequential implementation.  The &pooma;
+     Toolkit and a message-passing library automatically perform all
+     computation.</para>
+ 
+     <para>The command to run the programs is dependent on the run-time
+     system.  To use &mpi; with the Irix 6.5 operating system, one
+     can use the <command>mpirun</command> command.  For example,
+     <statement>mpirun -np 9 Doof2d-Array-distributed -mpi
+     --num-patches 3</statement> invokes the &mpi; run-time system with
+     nine processors.  The <statement>-mpi</statement> argument tells
+     the &pooma; executable <command>Doof2d-Array-distributed</command>
+     to use the &mpi; Library.  
+ 
+ HERE
+ 
+     The command <statement>Doof2d-Array-distributed -shmem -np 2
+     --num-patches 2</statement> 
+ 
+     To run <command>Doof2d-Array-distributed</command> with the &mm;
+     Shared Memory Library, use 
+ 
+ HERE</para>
+ 
+ <!-- HERE -->
+ 
+     <para>COMMENT: See <filename
+     class="libraryfile">background.html</filename> for a partial
+     explanation.  A <firstterm>context</firstterm> is a distinct
+     region of memory in some computer.  Execution thread is associated
+     with each context.  One or more different processors can be
+     associated with the same context.</para>
+ 
+     <para>QUESTION: How do &pooma; parallel concepts compare with
+     Fortran D or high-performance Fortran FINISH CITE:
+     {koelbel94:_high_perfor_fortr_handb}?</para>
+ 
+     <para>QUESTION: What does Cheetah do for us?  Must configure with
+     --messaging and Cheetah library must be available.  When running
+     Doof2d benchmark, use --num-patches N.  On LinuxKCC, use
+     '--num-patches p --run-impls 14 --sim-params N 0 1'.  Runtime
+     system must also provide some support.  How do I write about this?
+     What is an example?  How does one install Cheetah?</para>
+ 
+ <!-- HERE -->
+    </section>
+ 
+ 
+    <section id="tutorial-relations">
+     <title>Relations</title>
+ 
+     <para>UNFINISHED</para>
+ 
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="concepts">
+    <title>Overview of &pooma; Concepts</title>
+ 
+    <para>Describe the software application layers similar to
+    <filename>papers/SCPaper-95.html</filename> and "Short Tour of
+    &pooma;" in <filename>papers/SiamOO98_paper.ps</filename>.
+    Section 2.2, "Why a Framework?," of
+    <filename>papers/pooma.ps</filename> argues why a layered approach
+    eases use.  Section 3.1, "Framework Layer Description,"
+    describes the five layers.</para>
+ 
+    <para>FINISH: Write short glossary entries for each of these.</para>
+ 
+    <para>FINISH: Look through the source code to ensure all main
+    concepts are listed.</para>
+ 
+    <para>Here are (preliminary) &pooma; equations:</para>
+ 
+    <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	  orient="port" pgwide="0">
+     <title>&pooma; <quote>Equations</quote></title>
+ 
+     <tgroup cols="1" align="center">
+      <tbody>
+       <row>
+        <entry><para>field = data + materials + centering + layout + mesh</para></entry>
+        </row>
+       <row>
+        <entry><para>map from space to values</para></entry>
+       </row>
+       <row>
+        <entry><para>array = data + layout</para></entry>
+        </row>
+       <row>
+        <entry><para>map from indices to values</para></entry>
+       </row>
+       <row>
+        <entry><para>mesh = layout + origin + spacings</para></entry>
+        </row>
+       <row>
+        <entry><para>distribute domain through physical space</para></entry>
+       </row>
+       <row>
+        <entry><para>layout = domain + partition + layout_tag (distributed/replicated)</para></entry>
+        </row>
+       <row>
+        <entry><para>distribute domain's blocks among processors/contexts</para></entry>
+       </row>
+       <row>
+        <entry><para>partition = blocks + guard layers</para></entry>
+        </row>
+       <row>
+        <entry><para>split domain into blocks</para></entry>
+       </row>
+       <row>
+        <entry><para>domain = newDomain</para></entry>
+        </row>
+       <row>
+        <entry><para>space of permissible indices</para></entry>
+       </row>
+      </tbody>
+     </tgroup>
+    </table>
+ 
+ 
+    <para>FINISH: Following is a first try at describing the &pooma;
+    abstraction layers.  See also paper illustration.</para>
+ 
+    <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	  orient="port" pgwide="0">
+     <title>&pooma; Abstraction Layers</title>
+ 
+     <tgroup cols="1" align="center">
+      <tbody>
+       <row>
+        <entry><para>application program</para></entry>
+       </row>
+       <row>
+        <entry><para>&array; &field; (should have
+        <type>FieldEngine</type> under it)</para></entry>
+       </row>
+       <row>
+        <entry><para>&engine;</para></entry>
+       </row>
+       <row>
+        <entry><para>evaluators</para></entry>
+       </row>
+      </tbody>
+     </tgroup>
+    </table>
+ 
+    <para>FINISH: How does parallel execution fit in?</para>
+ 
+    <para>FINISH: Should we also name and describe each layer?</para>
+ 
+    <section id="concepts-domains">
+     <title>Domains</title>
+ 
+     <blockquote>
+      <attribution>Section 4 "Future Improvements in
+      &pooma; II" of
+      <filename>papers/SiamOO98_paper.ps</filename></attribution>
+      <para>
+      A &domain; is a set of discrete points in some space.…
+      &domain;s provide all of the expected domain calculus
+      capabilities such as subsetting and intersection.
+      </para>
+     </blockquote>
+ 
+     <para>Section 3, "Domains and Views," of
+     <filename>papers/iscope98.pdf</filename> describes five types of
+     domains</para>
+    </section>
+ 
+ 
+    <section id="concepts-layouts">
+     <title>Layouts</title>
+ 
+     <para>UNFINISHED</para>
+ 
+     <para>Also describe partitions and guard cells within here.</para>
+ 
+    </section>
+ 
+ 
+    <section id="concepts-meshes">
+     <title>Meshes</title>
+ 
+     <para>UNFINISHED</para>
+    </section>
+ 
+ 
+    <section id="concepts-data_parallel">
+     <title>Data-Parallel Statements</title>
+ 
+     <para>Can we use "An Overview of &pete;" from
+     <filename>papers/PETE_DDJ/ddj_article.html</filename> or is this
+     too low-level?</para>
+ 
+     <para>Section 3.2.1 of <filename>papers/pooma.ps</filename>
+     gives a simple example of data-parallel expression.  It also has a
+     paragraph introducing data-parallel operations and selecting
+     subsets of domains.  Section 3.4 describes the Chained
+     Expression Object (<acronym>CEO</acronym>), apparently a precursor
+     of &pete;.  Regardless, it provides some motivation and
+     introductory material.</para>
+ 
+     <para>From Section 4 of
+     <filename>papers/SiamOO98_paper.ps</filename>:</para>
+ 
+     <para>This version of &pete; reduces compile time of user codes
+     and utilizes compile-time knowledge of expression &domain;s for
+     better optimization.  For example, more efficient loops for
+     evaluating an expression can be generated if &pete; knows that the
+     &domain; has unit stride in memory.</para>
+ 
+     <para>Section 4, "Expressions and Evaluators", of
+     <filename>papers/iscope98.pdf</filename> has a good explanation of
+     &pooma; II's expression trees and expression engines.</para>
+ 
+     <para>COMMENT: <filename
+ 			     class="libraryfile">background.html</filename> has some related
+     &pete; material.</para>
+    </section>
+ 
+    <section id="concepts-containers">
+     <title>Containers</title>
+ 
+     <section id="concepts-containers-array">
+      <title>&array;</title>
+ 
+      <blockquote>
+       <attribution>Section 4 "Future Improvements in
+      &pooma; II" of
+      <filename>papers/SiamOO98_paper.ps</filename></attribution>
+       <para>
+      An &array; can be thought of as a map from one &domain; to
+      another.… &array;s depend only on the interface of
+      &domain;s.  Thus, a subset of view of an &array; can be
+      manipulated in all the same ways as the original &array;.
+      &array;s can perform indirect addressing because the output
+      &domain; one one &array; can be used as the input &domain; of
+      another &array;.  &array;s also provide individual element
+      access.</para>
+      </blockquote>
+ 
+ 
+      <para>
+       (unformatted) From
+       <filename>papers/GenericProgramming_CSE/dubois.html</filename>:
+       The &pooma; &array; concept provides an example of how these
+       generic-programming features can lead to flexible and efficient
+       code. An Array maps a fairly arbitrary input domain to an
+       arbitrary range of outputs. When used by itself, an &array;
+       object <varname>A</varname> refers to all of the values in its
+       domain. Element-wise mathematical operations or functions can be
+       applied to an array using straightforward notation, like A + B
+       or sin(A). Expressions involving Array objects are themselves
+       Arrays. The operation A(d), where d is a domain object that
+       describes a subset of A's domain, creates a view of A that
+       refers to that subset of points. Like an array expression, a
+       view is also an Array. If d represents a single point in the
+       domain, this indexing operation returns a single value from the
+       range. Equivalently, one can index an N-dimensional Array by
+       specifying N indices, which collectively specify a single point
+       in the input domain: A(i1, i2, ..., iN).</para>
+ 
+      <para>The &pooma; multi-dimensional Array concept is similar to
+       the Fortran 90 array facility, but extends it in several
+       ways. Both &pooma; and Fortran arrays can have up to seven
+       dimensions, and can serve as containers for arbitrary
+       types. Both support the notion of views of a portion of the
+       array, known as array sections in F90. The &pooma; Array concept
+       supports more complex domains, including bounded, continuous
+       (floating-point) domains. Furthermore, Array indexing in &pooma;
+       is polymorphic; that is, the indexing operation X(i1,i2) can
+       perform the mapping from domain to range in a variety of ways,
+       depending on the particular type of the Array being
+       indexed.</para>
+ 
+      <para>Fortran arrays are dense and the elements are arranged
+       according to column-major conventions. Therefore, X(i1,i2)
+       refers to element number i1-1+(i2-1)*numberRowsInA. However, as
+       Fig. 1 shows, Fortran-style "Brick" storage is not the only
+       storage format of interest to scientific programmers. For
+       compatibility with C conventions, one might want to use an array
+       featuring dense, row-major storage (a C-style Brick). To save
+       memory, it might be advantageous to use an array that only
+       stores a single value if all its element values are the
+       same. Other sparse storage schemes that only store certain
+       values may also be desirable. To exploit parallelism, it is
+       convenient for an array's storage to be broken up into patches,
+       which can be processed independently by different CPUs. Finally,
+       one can imagine an array with no data at all. For example, the
+       values can be computed from an expression involving other
+       arrays, or analytically from the indices.
+      </para>
+ 
+      <para>The &pooma; &array; Class Template</para>
+ 
+      <para>Next we describe &pooma;'s model of the Array concept, the
+      Array class template. The three most important requirements from
+      the point of view of overall design are: (1) arbitrary domain,
+      (2) arbitrary range, and (3) polymorphic indexing. These express
+      themselves in the template parameters for the &pooma; Array
+      class. The template
+      <programlisting>
+      template <int Dim, class T = double, class EngineTag = Brick>
+      class Array;
+      </programlisting>
+      is a specification for creating a set of classes all named
+      Array. The template parameters Dim, T, and EngineTag determine
+      the precise type of the Array. Dim represents the dimension of
+      the array's domain. T gives the type of array elements, thereby
+      defining the output range of the array. EngineTag specifies the
+      the manner of indexing and types of the indices.</para>
+ 
+      <para>End From
+      <filename>papers/GenericProgramming_CSE/dubois.html</filename>:</para>
+ 
+      <para>Section 2, "Arrays and Engines," of
+      <filename>papers/iscope98.pdf</filename> describes both &array;s
+      and &engine;s.  This may or may not duplicate the material in
+      <filename>papers/GenericProgramming_CSE/dubois.html</filename>.</para>
+ 
+      <section id="concepts-containers-array-views">
+       <title>Views of &array;s</title>
+ 
+       <para>Section 3, "Domains and Views," of
+       <filename>papers/iscope98.pdf</filename> motivates the need for
+       views:
+        <blockquote>
+ 	<para>One of the primary uses of domains is to specify
+ 	subsections of &array; objects.  Subarrays are a common
+ 	feature of array classes; however, it is often difficult to
+ 	make such subarrays behave like first-class objects.  The
+ 	&pooma; II engine concept provides a clean solution to
+ 	this problem: subsetting an &array; with a domain object
+ 	creates a new &array; that has a view engine.</para>
+        </blockquote>
+       </para>
+      </section>
+     </section>
+ 
+     <section id="concepts-containers-field">
+      <title>&field;</title>
+ 
+      <para>QUESTION: Do we include boundary conditions here?
+ 
+      FINISH: Do we have an example that shows something not possible
+      with &array;?</para>
+ 
+      <para>Describe and illustrate multi-material and
+      multivalue?</para>
+ 
+      <para>ADD: description of meshes and guard layers.</para>
+ 
+     </section>
+ 
+ 
+     <section id="concepts-containers-matrix">
+      <title><type>TinyMatrix</type></title>
+ 
+      <para>Section 3.2.2 of
+      <filename>papers/pooma.ps</filename> describes &vector;s and
+      matrix classes.</para>
+     </section>
+    </section>
+ 
+    <section id="concepts-engines">
+     <title>Engines</title>
+ 
+     <para>(unformatted) From
+     <filename>papers/GenericProgramming_CSE/dubois.html</filename>:</para>
+ 
+     <para>The Engine Concept</para>
+ 
+     <para>To implement polymorphic indexing, the Array class defers
+     data storage and data lookup to an engine object. The requirements
+     that the Array template places on its engine provide the
+     definition for the Engine concept. We'll describe these by
+     examining a simplified version of the Array template, shown in
+     Fig. 2.</para>
+ 
+     <para>First, the Array class determines and exports (makes
+     Engine_t part of Array's public interface) the type of the engine
+     class that it will use:
+     <programlisting>
+     typedef Engine<Dim, T, EngineTag> Engine_t;
+     </programlisting>
+     This statement declares Engine_t to be an alias for the type
+     Engine<Dim,T,EngineTag>. This is the first requirement
+     placed on engine classes: they must be specializations of a
+     general Engine template whose template parameters are identical to
+     those of Array. Next, the Array template determines the type of
+     scalar arguments (indices) to be used in operator(), the function
+     that implements &pooma;'s Fortran-style indexing syntax X(i1,i2):
+     <programlisting>
+     typedef typename Engine_t::Index_t Index_t;
+     </programlisting>
+     This statement defines another type alias:
+     Array<Dim,T,EngineTag>::Index_t is simply an alias for
+     Engine_t::Index_t. Engine_t::Index_t is a qualified name, which
+     means that the type Index_t is found in the class Engine_t. This
+     is the second requirement for the Engine concept: the class
+     Engine_t must define a public type called Index_t. This line will
+     not compile if that definition is not supplied. This indirection
+     is one of the ways that &pooma; supports polymorphic indexing. If
+     the Engine works with a discrete integer domain, it defines its
+     Index_t to be an integral type. If the Engine works in a
+     continuous domain, it defines its Index_t to be a floating-point
+     type.</para>
+ 
+     <para>The data lookup is performed in the operator() function. We
+     see that Array simply passes the indices on to its engine
+     object. Thus, we have the third requirement for the Engine
+     concept: it must provide a version of operator() that takes Dim
+     values of type Index_t.</para>
+ 
+     <para>Simply passing the indices on to the engine object may seem
+     odd. After all, engine(i,j) looks like we're just indexing another
+     array. There are several advantages to this extra level of
+     indirection. The Array class is as faithful a model of the Array
+     concept as possible, while the Engine class is a low-level
+     interface to a user-defined data source. As a result, Array has a
+     wide variety of constructors for user convenience, while engines
+     have but a few. Array supports a wide variety of overloaded
+     operator() functions for view creation and indexing. Engines
+     support indexing only. Array does not have direct access to the
+     data, which is managed by the engine object. Finally, Array has a
+     wide variety of overloaded mathematical operators and functions,
+     and works with the Portable Expression Template Engine (PETE) [4]
+     to provide efficient evaluation of Array expressions. Engines have
+     no such support. In general, Array is much more complex and
+     feature-laden than Engine. This is the prime advantage of the
+     separation of interface and implementation: Array only has to be
+     implemented once by the &pooma; developers. Engines are simple
+     enough to be written by users and plugged directly into the Array
+     framework.</para>
+ 
+     <para>Figure 3 illustrates the "Brick" specialization of the
+     Engine template, which implements Fortran-style lookup into a
+     block of memory. First, there is the general Engine template,
+     which is empty as there is no default behavior for an unknown
+     EngineTag. The general template is therefore not a model for the
+     Engine concept and Array classes attempting to use it will not
+     compile. Next, there is the definition of the Brick class, a
+     policy tag whose sole purpose is to select a particular
+     specialization of the Engine template. Finally, there is the
+     partial specialization of the Engine template. Examining its body,
+     we see the required Index_t typedef and the required operator(),
+     which follows the Fortran prescription for generating an offset
+     into the data block based on the row, column, and the number of
+     rows. All of the requirements are met, so the Brick-Engine class
+     is a model of the Engine concept.</para>
+ 
+     <para>End From
+     <filename>papers/GenericProgramming_CSE/dubois.html</filename>:</para>
+ 
+     <para>(unformatted) From
+     <filename>papers/GenericProgramming_CSE/dubois.html</filename>:</para>
+ 
+     <para>Compile-time Versus Run-Time Polymorphism</para>
+ 
+     <para>Encapsulating the indexing in an Engine class has important
+     advantages, both in terms of flexibility and efficiency. To
+     illustrate this point, we introduce the PolarGaussian-Engine
+     specialization in Fig. 4. This is an analytic engine that
+     calculates its values directly from its inputs. Unlike the
+     Brick-Engine, this engine is "indexed" with data of the same type
+     as its output: it maps a set of T's to a single T. Therefore, the
+     Index_t typedef selects T as the index type, as opposed to the int
+     in the Brick-Engine specialization. The operator() function also
+     differs in that it computes the return value according to an
+     analytic formula.</para>
+ 
+     <para>Both Engine<Dim,T,Brick> and
+     Engine<Dim,T,PolarGaussian> can be plugged in to an Array by
+     simply varying the Array's EngineTag. This is possible despite the
+     fact that the two classes exhibit dramatically different behavior
+     because they are both models of the Engine concept.</para>
+ 
+     <para>Notice that we have achieved polymorphic indexing without
+     the use of inheritance or virtual functions. For instance,
+     consider the following code snippet:
+     <programlisting>
+     Array<2, double, Brick> a;
+     Array<2, double, PolarGaussian> b;
+ 
+     double x = a(2, 3);     // x = a.engine.data[2 + 3 * a.engine.numRows];
+     double y = b(2.0, 3.0); // y = exp(-(2.0*2.0+3.0*3.0) / b.engine.delta);
+     </programlisting>
+     The data lookup functions for the two Arrays perform completely
+     different operations. Since this is accomplished using static
+     types, it is known as compile-time polymorphism. Moreover,
+     everything is known at compile time, so the functions are fully
+     inlined and optimized, thereby yielding code equivalent to that
+     shown in the comments above.</para>
+ 
+     <para>The flexibility and efficiency of compile-time polymorphism
+     cannot be duplicated with a run-time implementation. To illustrate
+     this point, in Fig. 5, we re-implement our Array concept using the
+     classic Envelope-Letter pattern [5], with the array class,
+     RTArray, being the envelope and the run-time-engine, RTEngine,
+     being the letter. RTArray defers data lookup to the engine object
+     by invoking the engine's functions through a pointer to the
+     RTEngine base class. Figure 6 illustrates the RTEngine base class
+     and Fig. 7 illustrates two descendants: RTBrick and
+     RTPolarGaussian.</para>
+ 
+     <para>The run-time implementation provides the same basic
+     functionality as the compile-time implementation, but it is not as
+     flexible or as efficient. It lacks flexibility in that the return
+     type of the indexing operation must be specified in the RTEngine
+     base class and in the RTArray class. Thus, in Figs. 5 and 6,we see
+     versions of RTArray::operator() and RTEngine::index functions that
+     take both int's and T's. If the programmer wants to add another
+     index-type option, these classes must be modified. This is a
+     violation of the open-closed principle proposed by Meyer
+     [6]. Also, since RTEngine descendants will usually only implement
+     one version of index, we cannot make RTEngine an abstract base
+     class. Instead, we have the default versions of index throw an
+     exception. Thus, compile-time error checking is
+     weakened. Furthermore, since indexing is done via a virtual
+     function call, it will almost never be inlined, which is not
+     acceptable in most scientific applications.</para>
+ 
+     <para>There are advantages to the Envelope-Letter approach. First,
+     all RTArray objects have the same type, allowing them to be stored
+     in homogeneous collections. This can simplify the design of some
+     applications. Second, RTArray objects can change their engines at
+     runtime, and thus effectively change their types on the fly??this
+     is the primary reason for using the Envelope-Letter idiom, and can
+     be very important in some applications.</para>
+ 
+     <para>For most scientific applications, however, these issues are
+     minor, and maximum performance for array indexing is of paramount
+     importance. Our compile-time approach achieves this performance
+     while providing the desired polymorphic indexing.</para>
+ 
+     <para>From Section 4 of
+     <filename>papers/SiamOO98_paper.ps</filename>:</para>
+ 
+     <para>The &array; class is templated on an &engine; type that
+     handles the actual implementation of the mapping from input to
+     output.  Thus, the &array; interface features are completely
+     separate from the implementation, which could be a single &c;
+     array, a function of some kind or some other mechanism.  This
+     flexibility allows an expression itself to be viewed through the
+     &array; interface.  Thus, one can write something like
+     <programlisting>
+     foo(A*B+C);
+     </programlisting> where <varname>A</varname>, <varname>B</varname> and
+     <varname>C</varname> are &array;s and <function>foo</function> is
+     a function taking an &array; as an argument.  The expression
+     <statement><varname>A</varname>*<varname>B</varname>+<varname>C</varname></statement>
+     will only be evaluated by the expression engine as needed by
+     <function>foo</function>.</para>
+ 
+     <para>In fact, one can even write &engine;s which are wrappers
+     around external data structures created in non-&pooma; codes and
+     know to manipulate these structures.  Once this is done, the
+     external entities have access to the entire &array; interface and
+     can utilize all of the powerful features of
+     &pooma; II.</para>
+ 
+     <para>Section 2, "Arrays and Engines," of
+     <filename>papers/iscope98.pdf</filename> describes both &array;s
+     and &engine;s.  This may or may not duplicate the material in
+     <filename>papers/GenericProgramming_CSE/dubois.html</filename>.</para>
+ 
+     <para>Section 4, "Expressions and Evaluators", of
+     <filename>papers/iscope98.pdf</filename> has a good explanation of
+     &pooma; II's expression trees and expression engines.</para>
+ 
+     <variablelist>
+      <varlistentry><term><type>MultiPatch</type> Engine</term>
+       <listitem><para>From <filename
+       class="libraryfile">README</filename>: To actually use multiple
+       contexts effectively, you need to use the MultiPatch engine with
+       patch engines that are Remote engines.  Then the data will be
+       distributed across multiple contexts instead of being copied on
+       every context.  See the files in example/Doof2d for a simple
+       example that creates a MultiPatch array that can be distributed
+       across multiple contexts and performs a stencil computation on
+       that array.</para></listitem>
+      </varlistentry>
+     </variablelist>
+    </section>
+ 
+ 
+    <section id="concepts-relations">
+     <title>Relations</title>
+ 
+     <para>UNFINISHED</para>
+    </section>
+ 
+ 
+    <section id="concepts-stencils">
+     <title>Stencils</title>
+ 
+     <para>Section 3.5.4, "Stencil Objects," of
+     <filename>papers/pooma.ps</filename> provides a few uses of
+     stencils.</para>
+ 
+     <para>Section 5, "Performance," of
+     <filename>papers/iscope98.pdf</filename> motivates and explains
+     stencils.</para>
+    </section>
+ 
+ 
+    <section id="concepts-contexts">
+     <title>Contexts</title>
+ 
+     <blockquote>
+      <attribution><filename
+      class="libraryfile">background.html</filename></attribution>
+      <para>In order to be able to cope with the variations in machine
+      architecture noted above, &pooma;'s distributed execution model
+      is defined in terms of one or more contexts, each of which may
+      host one or more threads. A context is a distinct region of
+      memory in some computer. The threads associated with the context
+      can access data in that memory region and can run on the
+      processors associated with that context. Threads running in
+      different contexts cannot access memory in other contexts.</para>
+ 
+      <para>A single context may include several physical processors,
+      or just one. Conversely, different contexts do not have to be on
+      separate computers—for example, a 32-node SMP computer could
+      have up to 32 separate contexts. This release of &pooma; only
+      supports a single context for each application, but can use
+      multiple threads in the context on supported platforms. Support
+      for multiple contexts will be added in an upcoming
+      release.</para>
+     </blockquote>
+    </section>
+ 
+ 
+    <section id="concepts-utilities">
+     <title>Utility Types: ???TITLE??</title>
+ 
+     <section id="concepts-containers-vector">
+      <title>&vector;</title>
+ 
+      <para>Section 3.2.2 of
+      <filename>papers/pooma.ps</filename> describes &vector;s and
+      matrix classes.</para>
+     </section>
+ 
+    </section>
+   </chapter>
+ 
+ 
+   <chapter id="sequential">
+    <title>Writing Sequential Programs</title>
+ 
+    <para>UNFINISHED</para>
+ 
+    <section id="sequential-benchmarks">
+     <title>&benchmark; Programs</title>
+ 
+     <para>Define a &benchmark; program vs. an example or an
+     executable.  Provide a short overview of how to run these
+     programs.  Provide an overview of how to write these programs.
+     See <filename
+ 		  class="headerfile">src/Utilities/Benchmark.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="sequential-inform">
+     <title>Using <type>Inform</type>s for Output</title>
+ 
+     <para>UNFINISHED</para>
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="parallel">
+    <title>Writing Distributed Programs</title>
+ 
+    <para>Discuss the distributed model and guard cells.  See <filename
+    class="libraryfile">docs/parallelism.html</filename>.</para>
+ 
+    <para>Does any of the parallel implementation described in
+    <filename>papers/SCPaper-95.html</filename> still apply?</para>
+ 
+    <para>?Tuning program for maximize parallel performance?</para>
+ 
+    <para>external references to &mpi; and threads</para>
+ 
+    <para>QUESTION: Are there interesting, short parallel programs in
+    any &mpi; book that we can convert to &pooma;?</para>
+ 
+    <section>
+     <title>Layouts</title>
+ 
+     <para>An out-of-date description can be found in Section 3.3,
+     especially 3.3.2, of <filename>papers/pooma.ps</filename>
+     describes the global/local interactions and parallel abstraction
+     layers.</para>
+    </section>
+ 
+    <section>
+     <title>Parallel Communication</title>
+ 
+     <para>An out-of-date description can be found in
+      Section 3.3.3 of <filename>papers/pooma.ps</filename></para>
+    </section>
+ 
+    <section>
+     <title>Using Threads</title>
+ 
+     <para>QUESTION: Where do threads fit into the manual?  Do threads
+     even work?</para>
+ 
+     <para>From Section 4, of
+     <filename>papers/SiamOO98_paper.ps</filename></para>
+ 
+     <para>&pooma; II will make use of a new parallel run-time
+     system called &smarts; that is under development at the ACL.
+     &smarts; supports lightweight threads, so the evaluator will be
+     able to farm out data communication tasks and the evaluation of
+     subsets of an expression to multiple threads, thus increasing the
+     overlap of communication and computation.  Threads will also be
+     available at the user level for situations in which a
+     task-parallel approach is deemed appropriate.</para>
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="under_the_hood">
+    <title>Under the Hood: How &pooma; Works</title>
+ 
+    <para>from point of view of &cc; interpreter</para>
+ 
+    <section id="under_the_hood-PETE">
+     <title>&pete;</title>
+ 
+     <para>Use the material in
+     <filename>papers/PETE_DDJ/ddj_article.html</filename>, which gives
+     example code and descriptions of how the code works.</para>
+ 
+     <para>See material in <filename
+     class="libraryfile">background.html</filename>'s <quote>Expression
+     Templates.</quote></para>
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="debugging_profiling">
+    <title>Debugging and Profiling &pooma; Programs</title>
+ 
+    <para>UNFINISHED</para>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="example">
+    <title>Example Program: Jacobi Solver</title>
+ 
+    <para>QUESTION: Is this chapter necessary?  Do we have enough
+    existing source code to write this chapter?</para>
+ 
+   </chapter>
+  </part>
+ 
+  <part id="reference">
+   <title>&pooma; Reference Manual</title>
+ 
+   <chapter id="tmp_ref">
+    <title>TMP: This Chapter Holds These Comments But Will Be Removed</title>
+ 
+    <para>For each template parameter need to describe the constraints
+    on it.</para>
+ 
+    <para>Remove this section when the following concerns have been
+    addressed.</para>
+ 
+    <para>Add a partintro explaining file suffixes such as <filename
+    class="headerfile">.h</filename>, <filename
+    class="libraryfile">.cpp</filename>, <filename
+    class="libraryfile">.cmpl.cpp</filename>, <filename
+    class="libraryfile">.mk</filename>, <filename
+    class="libraryfile">.conf</filename>.  Should we also explain use
+    of <literal>inline</literal> even when necessary and the template
+    model, <!-- FIXME: s/literal/keyword/ --> e.g., including <filename
+    class="libraryfile">.cpp</filename> files.</para>
+ 
+    <para>QUESTION: What are the key concepts around which to organize
+    the manual?</para>
+ 
+    <para>QUESTION: What format should the manual use?</para>
+ 
+    <blockquote>
+     <attribution>Musser, Derge, and Sanai, §20.0.</attribution>
+     <para>It is important to state the requirements on the components
+     as generally as possible.  For example, instead of saying
+     <quote>class <type>X</type> must define a member function
+     <function>operator++()</function>,</quote> we say <quote>for any
+     object <varname>x</varname> of type <type>X</type>,
+     <classname>++x</classname> is defined.</quote></para>
+    </blockquote>
+   </chapter>
+ 
+ 
+   <chapter id="class_structure_ref">
+    <title>A Typical &pooma; Class</title>
+ 
+    <variablelist>
+     <title>Class Member Notation</title>
+     <varlistentry>
+      <term><fieldsynopsis>
+        <varname>*_t</varname>
+       </fieldsynopsis>
+      </term>
+      <listitem>
+       <para>type within a class.  QUESTION: What is the &cc; name for
+       this?</para>
+      </listitem>
+     </varlistentry>
+     <varlistentry>
+      <term><fieldsynopsis>
+        <varname>*_m</varname>
+       </fieldsynopsis>
+      </term>
+      <listitem>
+       <para>data member</para>
+      </listitem>
+     </varlistentry>
+    </variablelist>
+ 
+    <variablelist>
+     <title>&pooma; Class Vocabulary</title>
+     <varlistentry>
+      <term>component</term>
+      <listitem>
+       <para>one of several values packaged together.  For example, a
+       three-dimensional vector has three components, i.e., three
+       values.</para>
+      </listitem>
+     </varlistentry>
+     <varlistentry>
+      <term>element-wise</term>
+      <listitem>
+       <para>applied to each element in the group, e.g., an array</para>
+      </listitem>
+     </varlistentry>
+     <varlistentry>
+      <term>reduction</term>
+      <listitem>
+       <para>repeated application of a binary operator to all elements,
+       yielding one value</para>
+      </listitem>
+     </varlistentry>
+     <varlistentry>
+      <term>tag</term>
+      <listitem>
+       <para>an enumerated value indicating inclusion in a particular
+       semantic class.  The set of values need not be explicitly
+       declared.</para>
+      </listitem>
+     </varlistentry>
+    </variablelist>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="installing_configuring_ref">
+    <title>Installing and Configuring &pooma;</title>
+ 
+    <orderedlist>
+     <listitem>
+      <para>Installing &pooma;.</para>
+     </listitem>
+     <listitem>
+      <para>Requirements for configuration files.</para>
+     </listitem>
+    </orderedlist>
+ 
+    <para>Include descriptions of using &smarts;, &cheetah;, τ,
+    &pdt;.</para>
+ 
+    <para>QUESTION: Does it install on windows and on mac?  If so, what
+    are the instructions?  See also <filename
+    class="libraryfile">INSTALL.{mac,unix,windows}.</filename></para>
+ 
+    <para><filename class="libraryfile">README</filename> has some
+    information on &cheetah; and threads in the <quote>Message-Based
+    Parallelism</quote> section.</para>
+ 
+    <para>Which additional packages are necessary and when?</para>
+ 
+    <para>What configure options should we list?  See <filename
+    class="libraryfile">configure</filename>.  Be sure to list
+    debugging option and how its output relates to <filename
+    class="libraryfile">config/LINUXgcc.suite.mk</filename>.</para>
+ 
+    <para><filename class="directory">config/arch</filename> has files
+    for (OS, compiler) pairs.  Explain how to modify a configuration
+    file.  List requirements when making a new configuration file (low
+    priority).</para>
+ 
+    <para><filename
+    class="libraryfile">config/LINUXgcc.suite.mk</filename> has output
+    from <filename class="libraryfile">configure</filename>.  Useful to
+    relate to configuration files and <filename
+    class="libraryfile">configure</filename>'s debugging output.</para>
+    
+   </chapter>
+ 
+ 
+   <chapter id="compilation_ref">
+    <title>Compilation and &make; Files</title>
+ 
+    <para>We assume Gnu make.  Do we know what assumptions are made?</para>
+ 
+    <para>How do all these files interact with each other?  Ala a make
+    interpreter, give an example of which files are read and
+    when.</para>
+ 
+    <variablelist>
+     <varlistentry><term><filename
+     class="libraryfile">config/Shared/README.make</filename></term>
+     <listitem><para>This has short descriptions of many files,
+     especially in <filename
+     class="directory">config/Shared</filename>.</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">makefile</filename></term>
+     <listitem><para>These appear throughout all directories.  What are
+     the equivalences classes and what are their
+     parts?</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">include.mk</filename></term>
+     <listitem><para>What does this do?  Occurs in many directories:
+     when?  Template seems to be <filename
+     class="libraryfile">config/Shared/include2.mk</filename>.</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">subdir.mk</filename></term>
+     <listitem><para>list of subdirectories; occurs in several
+     directories: when?  <filename
+     class="libraryfile">src/subdir.mk</filename> is a good
+     example.</para></listitem>
+     </varlistentry>
+     <varlistentry>
+      <term><filename class="libraryfile">objfile.mk</filename></term>
+      <listitem>
+       <para>list of object files to construct, presumably from
+       <filename>*.cmpl.cpp</filename> files.
+       <filename>src/Utilities/objfile.mk</filename> is an
+       example.</para>
+      </listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">config/Shared/rules.mk</filename></term>
+     <listitem><para>most compiler rules</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">config/head.mk</filename></term>
+     <listitem><para>read at beginning of each
+     makefile?</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">config/Shared/tail.mk</filename></term>
+     <listitem><para>read at end of each makefile?</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">config/Shared/variables.mk</filename></term>
+     <listitem><para>Is this used?</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">config/Shared/compilerules.mk</filename></term>
+     <listitem><para>table of origin and target suffixes and commands
+     for conversion</para></listitem>
+     </varlistentry>
+ 
+    </variablelist>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="arrays_ref">
+    <title>&array;s</title>
+ 
+    <para>Include <filename
+ 			   class="headerfile">src/Pooma/Arrays.h</filename> to use &array;s.
+    The implementation source code is in <filename
+ 						  class="directory">src/Array</filename>.</para>
+ 
+    <para>FINISH: Define an array.  Introduce its parts.</para>
+ 
+    <para>ADD: some mention of the maximum supported number of
+    dimensions somewhere.</para>
+ 
+    <section id="arrays_ref-array">
+     <title>The &array; Container</title>
+ 
+     <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	   orient="port" pgwide="0">
+      <title>Template Parameters</title>
+ 
+      <tgroup cols="2" align="left">
+       <thead>
+        <row>
+ 	<entry>Parameter</entry>
+ 	<entry>Interpretation</entry>
+        </row>
+       </thead>
+       <tbody>
+        <row>
+ 	<entry><varname>Dim</varname></entry>
+ 	<entry><para>dimension</para></entry>
+        </row>
+        <row>
+ 	<entry><varname>T</varname></entry>
+ 	<entry><para>array element type</para></entry>
+        </row>
+        <row>
+ 	<entry>EngineTag</entry>
+ 	<entry><para>type of computation engine object</para></entry>
+        </row>
+       </tbody>
+      </tgroup>
+     </table>
+ 
+     <para>QUESTION: How do I introduce class type definitions, when
+     they are used, i.e., compile-time or run-time, and when
+     programmers should use them?</para>
+ 
+     <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	   orient="port" pgwide="0">
+      <title>Compile-Time Types and Values</title>
+ 
+      <tgroup cols="2" align="left">
+       <thead>
+        <row>
+ 	<entry>Type or Value</entry>
+ 	<entry>Interpretation</entry>
+        </row>
+       </thead>
+       <tbody>
+        <row>
+ 	<entry><fieldsynopsis><varname>This_t</varname></fieldsynopsis></entry>
+ 	<entry><para>the &array; object's type</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>Engine_t</varname></fieldsynopsis></entry>
+ 	<entry><para>the &array; object's engine's type</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>EngineTag_t</varname></fieldsynopsis></entry>
+ 	<entry><para>indication of engine's category</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>Element_t</varname></fieldsynopsis></entry>
+ 	<entry><para>the type of the array elements, i.e., <type>T</type></para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>ElementRef_t</varname></fieldsynopsis></entry>
+ 	<entry><para>the type of a reference to an array element,
+ 	i.e., <type>T&</type>.  Equivalently, the type to write to a
+ 	single element.</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>Domain_t</varname></fieldsynopsis></entry>
+ 	<entry><para>the array's domain's type, i.e., the type of the
+ 	union of all array indices</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>Layout_t</varname></fieldsynopsis></entry>
+ 	<entry><para>unknown</para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>dimensions</varname></fieldsynopsis></entry>
+ 	<entry><para>integer equalling the number of dimensions, i.e.,
+ 	<varname>Dim</varname></para></entry>
+        </row>
+        <row>
+ 	<entry><fieldsynopsis><varname>rank</varname></fieldsynopsis></entry>
+ 	<entry><para>integer equalling the number of dimensions, i.e.,
+ 	<varname>Dim</varname>; a synonym for <fieldsynopsis>
+ 	   <varname>dimensions</varname></fieldsynopsis></para></entry>
+        </row>
+       </tbody>
+      </tgroup>
+     </table>
+ 
+     <section id="arrays_ref-array-constructors">
+      <title>Constructors and Destructors</title>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>Constructors and Destructors</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Function</entry>
+ 	 <entry>Effect</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <constructorsynopsis>
+ 	   <methodname>Array</methodname>
+ 	   <void></void>
+ 	  </constructorsynopsis>
+          </entry>
+ 	 <entry><para>Creates an array that will be resized
+ 	 later.</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>
+           <constructorsynopsis>
+ 	   <methodname>Array</methodname>
+ 	   <methodparam><modifier>const</modifier> <type>Engine_t&</type>
+ 	    <parameter>engine</parameter></methodparam>
+ 	  </constructorsynopsis>
+          </entry>
+ 	 <entry><para>Creates an array with an engine equivalent to
+          the <parameter>engine</parameter>.  This array will have the
+          same values as <parameter>engine</parameter>.  QUESTION: Why
+          would a user every want to use this
+          constructor?</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>
+           <constructorsynopsis>
+ 	   <methodname>Array</methodname>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type> Engine<Dim2, T2, EngineTag2>&</type>
+ 	    <parameter>engine</parameter>
+ 	   </methodparam>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type>Initializer&</type> <parameter>init</parameter>
+ 	   </methodparam>
+ 	  </constructorsynopsis>
+          </entry>
+ 	 <entry><para>What does this do?</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>ADD ALL CONSTRUCTORS AND DESTRUCTORS.</entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+     </section>
+ 
+ 
+     <section id="arrays_ref-array-initializers">
+      <title>Initializers</title>
+ 
+      <para>Add a table.</para>
+     </section>
+ 
+ 
+     <section id="arrays_ref-array-access">
+      <title>Element Access</title>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>&array; Element Access</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Function</entry>
+ 	 <entry>Effect</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>Element_t</type> <methodname>read</methodname>
+ 	   <void></void>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>unknown: See line 1839.</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>Element_t</type> <methodname>read</methodname>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type>Sub1&</type> <parameter>s1</parameter>
+ 	   </methodparam>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type> Sub2&</type> <parameter>s2</parameter>
+ 	   </methodparam>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>How does the version with template parameters,
+          e.g., <type>Sub1</type> differ from the <type>int</type>
+          version?</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>Element_t</type> <methodname>operator()</methodname>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type>Sub1&</type> <parameter>s1</parameter>
+ 	   </methodparam>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type>Sub2&</type> <parameter>s2</parameter>
+ 	   </methodparam>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>How does this differ from <statement>read(const
+           Sub1& s1, const Sub2& s2)</statement>?  </para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>ADD ALL <function>read</function>s and
+ 	 <function>operator()</function>s.</entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+     </section>
+ 
+ 
+     <section id="arrays_ref-array-component">
+      <title>Component Access</title>
+ 
+      <para>When an array stores elements having components, e.g., an
+      array of vectors, tensors, or arrays, the
+      <methodname>comp</methodname> returns an array consisting of the
+      specified components.  The original and component array share the
+      same engine so changing the values in one affects values in the
+      other.</para>
+ 
+      <para>For example, if &n; × &n; array <varname>a</varname>
+      consists of three-dimensional real-valued vectors,
+      <statement>a.comp(1)</statement> returns a &n; × &n;
+      real-valued array of all the middle vector components.  Assigning
+      to the component array will also modify the middle components of
+      the vectors in <varname>a</varname>.</para>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>&array; Component Access</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Function</entry>
+ 	 <entry>Effect</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>UNKNOWN compute this</type> <methodname>comp</methodname>
+ 	   <methodparam>
+ 	    <modifier>const</modifier>
+ 	    <type>int&</type>
+ 	    <parameter>i1</parameter>
+ 	   </methodparam>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>unknown: See line 1989.</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>ADD ALL <methodname>comp</methodname>s.</entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+     </section>
+ 
+     <section id="arrays_ref-array-accessors">
+      <title>Accessors</title>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>&array; Accessor Methods</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Function</entry>
+ 	 <entry>Effect</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>int</type> <methodname>first</methodname>
+ 	   <methodparam>
+ 	    <type>int</type>
+ 	    <parameter>d</parameter>
+ 	   </methodparam>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>unknown: See line 2050</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>ADD ALL other accessor methods, including
+ 	 <methodname>engine</methodname>.</entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+     </section>
+ 
+ 
+     <section id="arrays_ref-array-copying">
+      <title>Copying &array;s</title>
+ 
+      <para>Explain how copied arrays and views of arrays share the
+      same underlying engine so changing values in one also affects the
+      other.  This is called a shallow copy.</para>
+     </section>
+ 
+ 
+     <section id="arrays_ref-array-utilities">
+      <title>Utility Methods</title>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>&array; Utility Methods</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Function</entry>
+ 	 <entry>Effect</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <methodsynopsis>
+ 	   <type>void</type> <methodname>makeOwnCopy</methodname>
+ 	   <void></void>
+ 	  </methodsynopsis>
+          </entry>
+ 	 <entry><para>unknown: See line 2044</para></entry>
+ 	</row>
+ 	<row>
+ 	 <entry>ADD ALL other utility methods.</entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+     </section>
+ 
+     
+     <section id="arrays_ref-array-implementation">
+      <title>Implementation Details</title>
+ 
+      <para>As a container, an &array;'s implementation is quite
+      simple.  Its <keywordname>private</keywordname>data consists of
+      an engine, and it has no <keywordname>private</keywordname>
+      functions.</para>
+ 
+      <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	    orient="port" pgwide="0">
+       <title>&array; Implementation Data</title>
+ 
+       <tgroup cols="2" align="left">
+        <thead>
+ 	<row>
+ 	 <entry>Data Member</entry>
+ 	 <entry>Meaning</entry>
+ 	</row>
+        </thead>
+        <tbody>
+ 	<row>
+ 	 <entry>
+           <fieldsynopsis>
+ 	   <modifier>private</modifier>
+ 	   <type>Engine_t</type> <varname>engine_m</varname>
+ 	  </fieldsynopsis>
+          </entry>
+ 	 <entry><para>engine computing the array's values</para></entry>
+ 	</row>
+        </tbody>
+       </tgroup>
+      </table>
+ 
+     </section>
+    </section>
+ 
+ 
+    <section id="arrays_ref-dynamicarray">
+     <title>&dynamicarray;s: Dynamically-Sized Domains</title>
+ 
+     <para>A DynamicArray is a read-write array with extra
+     create/destroy methods.  It can act just like a regular Array, but
+     can have a dynamically-changing domain.  See <filename
+ 							   class="headerfile">src/DynamicArray/DynamicArray.h</filename>.</para>
+ 
+     <para>ADD: Briefly describe what the class does and an example of
+     where it is used.</para>
+ 
+     <para>ADD: Check that its interface is actually the same as for
+     &array;.</para>
+ 
+     <para>ADD: Check that the operations on dynamic arrays are
+     actually the same as for &array;.  See <filename
+ 						     class="headerfile">src/DynamicArray/DynamicArrayOperators.h</filename>,
+     <filename
+ 	      class="headerfile">src/DynamicArray/PoomaDynamicArrayOperators.h</filename>,
+     and <filename
+ 		  class="headerfile">src/DynamicArray/VectorDynamicArrayOperators.h</filename>.</para>
+ 
+     
+     <section id="arrays_ref-dynamicarray-implementation">
+      <title>Implementation Details</title>
+ 
+      <para><type>DynamicArray</type> has no
+      <keywordname>protected</keywordname> or
+      <keywordname>private</keywordname> members.</para>
+     </section>
+    </section>
+ 
+ 
+    <section id="arrays_ref-views">
+     <title>Views of &array;s</title>
+ 
+     <para>UNFINISHED</para>
+    </section>
+ 
+ 
+    <section id="arrays_ref-assignment">
+     <title>&array; Assignments</title>
+ 
+     <para>&pooma; supports assignments to &array;s of other &array;s
+     and scalar values.  QUESTION: Is the following correct?  For the
+     former, the right-hand side array's domain must be at least as
+     large as the left-hand side array's domain.  Corresponding values
+     are copied.  Assigning a scalar value to an array ensures all the
+     array elements have the same scalar value.</para>
+ 
+     <para>UNFINISHED: Add a table containing assignment operators
+     found one lines 2097–2202.</para>
+    </section>
+ 
+ 
+    <section id="arrays_ref-print">
+     <title>Printing &array;s</title>
+ 
+     <para>&array;s support output to but not input from IO streams.
+     In particular, output to <type>ostream</type>s and file streams is
+     supported.</para>
+ 
+     <para>Add a table, using <filename
+     class="headerfile">src/Array/Array.h</filename>, lines
+     2408–2421.  See the implementation in <filename
+     class="headerfile">src/Array/PrintArray.h</filename>.</para>
+ 
+     <para>QUESTION: How does one print a &dynamicarray;.</para>
+    </section>
+ 
+ 
+    <section>
+     <title>Expressions Involving &array;s</title>
+ 
+     <para>In &pooma;, expressions may contain entire &array;s.  That
+     is, &array;s are first-class objects with respect to expressions.
+     For example, given &array;s <varname>a</varname> and
+     <varname>b</varname>, the expression <statement>a + b</statement>
+     is equivalent to an array containing the element-wise sum of the
+     two arrays.</para>
+ 
+     <para>Any finite number of the operators listed below can be used
+     in an expression.  The precedence and order of operation is the
+     same as with ordinary built-in types.</para>
+ 
+     <para>QUESTION: Do &field;s also support the same set of
+     operations?</para>
+ 
+     <para>QUESTION: Some operations in <filename
+ 						 class="headerfile">src/Field/FieldOperators.h</filename> use both
+     &array; and &field;.  Do we list them here or in the &field;
+     section or both or somewhere else?</para>
+ 
+     <para>In the table below, &array; supplants the exact return types
+     because they are complicated and rarely need to be explicitly
+     written down.</para>
+ 
+     <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	   orient="port" pgwide="0">
+      <title>Operators on &array;</title>
+ 
+      <tgroup cols="2" align="left">
+       <thead>
+        <row>
+ 	<entry>Operator</entry>
+ 	<entry>Value</entry>
+        </row>
+       </thead>
+       <tbody>
+        <row>
+ 	<entry>
+           <funcsynopsis>
+ 	  <funcprototype>
+ 	   <funcdef>Array <function>acos</function></funcdef>
+ 	   <paramdef>const <parameter>Array<Dim,T,EngineTag>& a</parameter></paramdef>
+ 	  </funcprototype>
+ 	 </funcsynopsis>
+          </entry>
+ 	<entry><para>an array containing the element-wise inverse
+          cosine of the array <parameter>a</parameter></para></entry>
+        </row>
+        <row>
+ 	<entry>ADD ALL other operators appearing in <filename
+ 							      class="headerfile">src/Array/ArrayOperators.h</filename>,
+         <filename
+ 		  class="headerfile">src/Array/ArrayOperatorSpecializations.h</filename>,
+         <filename
+ 		  class="headerfile">src/Array/PoomaArrayOperators.h</filename>,
+         and <filename
+ 		      class="headerfile">src/Array/VectorArrayOperators.h</filename>.</entry>
+        </row>
+       </tbody>
+      </tgroup>
+     </table>
+ 
+     <para>FINISH: Write one or two examples or refer to ones
+     previously in the text.</para>
+    </section>
+ 
+ 
+    <section id="arrays_ref-reductions">
+     <title>Reducing All &array; Elements to One Value</title>
+ 
+     <para>These reduction functions repeatedly apply a binary
+     operation to all array elements to yield a value.  These functions
+     are similar to the Standard Template Library's
+     <function>accumulate</function> function.  For example,
+     <function>sum</function> repeatedly applies the binary plus
+     operator to all array elements, yielding the sum of all array
+     elements.</para>
+ 
+     <para>FINISH: What order of operation, if any, is
+     guaranteed?</para>
+ 
+     <para>FINISH: Add a table of the functions in <filename
+ 							    class="headerfile">src/Array/Reductions.h</filename>.</para>
+ 
+     <para>How does one use one's own binary function?  See <filename
+     class="headerfile">src/Engine/Reduction.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="arrays_ref-utilities">
+     <title>Utility Functions</title>
+ 
+     <section id="arrays_ref-utilities-compression">
+      <title>Compressed Data</title>
+ 
+      <para>Add a table containing
+      <function>elementsCompressed</function>,
+      <function>compressed</function>, <function>compress</function>,
+      and <function>uncompress</function>.</para>
+     </section>
+ 
+ 
+     <section id="arrays_ref-utilities-sizes">
+      <title>Centering Sizes and Number of Materials</title>
+ 
+      <para>ADD: a description of <function>numMaterials</function> and
+      <function>centeringSize</function> found in <filename
+      class="headerfile">src/Field/Field.h</filename>.  These functions
+      are meaningless for &array; but are provided for consistency with
+      &field;.</para>
+     </section>
+ 
+     <section id="arrays_ref-utilities-subfield">
+      <title>Obtaining Subfields</title>
+ 
+      <para>ADD: a description of <function>subField</function> found
+      in <filename class="headerfile">src/Field/Field.h</filename>.
+      This function, meaningless for &array;, is provided for
+      consistency with &field;.</para>
+     </section>
+    </section>
+ 
+ 
+    <section id="arrays_ref-tmp">
+     <title>TMP: What do we do with these …? Remove this
+     section.</title>
+ 
+     <para>QUESTION: Do we describe the &leaffunctor;s specialized for
+     &array;s in <filename
+     class="headerfile">src/Array/Array.h</filename> or in the &pete;
+     reference section?  What about the functions in <filename
+     class="headerfile">src/Array/CreateLeaf.h</filename>?</para>
+ 
+     <para>QUESTION: What is an <type>EngineFunctor</type>?  We
+     probably should describe it in an analogous way as for
+     &leaffunctor;s.</para>
+ 
+     <para>QUESTION: Where do we write about
+     <type>ExpressionTraits</type> for &array;s?</para>
+ 
+     <para>QUESTION: Do we describe the <type>ElementProperties</type>
+     specialization at this place or in its section?</para>
+ 
+     <para>QUESTION: Do we describe the <type>Patch</type>
+     specialization for &array;s (<filename
+     class="headerfile">src/Array/Array.h</filename>:1300) in this
+     place or in a section for patches?</para>
+    </section>
+   </chapter>
+ 
+ 
+   <chapter id="fields_ref">
+    <title>&field;s</title>
+ 
+    <para>An &array; is a set of values indexed by
+    coordinates, one value per coordinate.  It models the computer
+    science idea of an array.  Similarly, a &field; is a set of values
+    indexed by coordinate.  It models the mathematical and physical
+    idea of a field represented by a grid of rectangular cells, each
+    having at least one value.  A &field;'s functionality is a superset
+    of an &array;'s functionality because:
+     <itemizedlist>
+      <listitem>
+       <para>A &field; is distributed through space so one can compute
+       the distances between cells.</para>
+      </listitem>
+      <listitem>
+       <para>Each cell can hold multiple values.  For example, a
+       rectangular cell can have one value on each of its faces.</para>
+      </listitem>
+      <listitem>
+       <para>Multiple materials can share the same cell.  For example,
+       different values can be stored in the same cell for carbon,
+       oxygen, and nitrogen.</para>
+      </listitem>
+     </itemizedlist>
+    Also, &field;s' values can be related by relations.  Thus, if one
+    field's values change, a dependent field's values can be
+    automatically computed when needed.  FIXME: See also the unfinished
+    works chapter's entry concerning relations and arrays.</para>
+ 
+    <para>QUESTION: Should we add a picture comparing and contrasting
+    an array and a field?</para>
+ 
+    <para>QUESTION: How much structure can be copied from the &array;
+    chapter?</para>
+ 
+    <para>QUESTION: Where is <type>NewMeshTag</type>, defined in
+    <filename class="headerfile">src/Field/Field.h</filename>,
+    used?</para>
+ 
+    <para>QUESTION: Do we describe the &leaffunctor;s specialized for
+    &field;s in <filename
+    class="headerfile">src/Field/Field.h</filename> or in the &pete;
+    reference section?  Use the same decision for &array;s.</para>
+ 
+    <para>QUESTION: What do the structure and functions in <filename
+    class="headerfile">src/Field/Mesh/PositionFunctions.h</filename>
+    do?</para>
+ 
+ 
+    <section id="fields_ref-field">
+     <title>The &field; Container</title>
+ 
+     <para>ADD: table of template parameters and table of compile-time
+     types and values.</para>
+ 
+ 
+     <section id="fields_ref-field-constructors">
+      <title>Constructors and Destructors</title>
+ 
+      <para>ADD: this section similar to &array;s's constructor and
+      destructor section.</para>
+     </section>
+ 
+     <section id="fields_ref-field-initializers">
+      <title>Initializers</title>
+ 
+      <para>Add a table.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-access">
+      <title>Element Access</title>
+ 
+      <para>ADD: a table ala &array;.  Be sure to include
+      <methodname>all</methodname>.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-component">
+      <title>Component Access</title>
+ 
+      <para>ADD: a table ala &array;.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-subfields">
+      <title>Obtaining Subfields</title>
+ 
+      <para>ADD: discussion and a table listing ways to obtain
+       subfields.  Although the implementation may treat subfield views
+       and other field views similarly (?Is this true?), they are
+       conceptually different ideas so we present them
+       separately.</para>
+ 
+      <para>See <filename
+ 			 class="headerfile">src/Field/Field.h</filename>'s
+       <methodname>operator[]</methodname>,
+       <methodname>subField</methodname>, …,
+       <methodname>material</methodname>.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-relations">
+      <title>Supporting Relations</title>
+ 
+      <para>ADD: a table with the member functions including
+      <methodname>addRelation</methodname>,
+      <methodname>removeRelations</methodname>,
+      <methodname>applyRelations</methodname>, and
+      <methodname>setDirty</methodname>.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-accessors">
+      <title>Accessors</title>
+ 
+      <para>ADD: a table using lines like <filename
+      class="headerfile">src/Field/Field.h</filename>:1243–1333.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-utilities">
+      <title>Utility Methods</title>
+ 
+      <para>ADD: a table including
+       <methodname>makeOwnCopy</methodname>.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-field-implementation">
+      <title>Implementation Details</title>
+ 
+      <para>ADD: a table similar to &array;'s.</para>
+ 
+     </section>
+ 
+    </section>
+ 
+ 
+    <section id="engines_ref-views">
+     <title>Views of &field;s</title>
+ 
+     <para>Be sure to relate to &array; views.  Note only three
+     dimensions are supported.</para>
+ 
+     <para>Be sure to describe <statement>f[i]</statement>.  Does this
+     refer to a particular material or a particular value within a
+     cell?  I do not remember.  See <type>SubFieldView</type> in
+     <filename class="headerfile">src/Field/Field.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-assignment">
+     <title>&field; Assignments</title>
+ 
+     <para>ADD: Describe supported assignments, relating to &array;'s
+     assignments.</para>
+ 
+     <para>UNFINISHED: Add a table containing assignment operators
+     found on <filename
+ 		       class="headerfile">src/Field/Field.h</filename>:2097–2202
+     and 1512–1611.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-print">
+     <title>Printing &field;s</title>
+ 
+     <para>QUESTION: How similar is this to printing &array;s?</para>
+ 
+     <para>&field;s support output to but not input from IO streams.
+     In particular, output to <type>ostream</type>s and file streams is
+     supported.</para>
+ 
+     <para>Add a table, using <filename
+ 				       class="headerfile">src/Field/Field.h</filename>, lines
+     1996–2009.  See the implementation in <filename
+ 							  class="headerfile">src/Field/PrintField.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-reductions">
+     <title>Combining &field; Elements</title>
+ 
+     <para>Like &array;s, &field;s support reduction of all elements to
+     one value.  Additionally, the latter supports computing a field's
+     values using field stencils.  QUESTION: How do I describe this
+     with a minimum of jargon?</para>
+ 
+     <para>ADD: something similar to &array; reductions.</para>
+ 
+     <para>FINISH: Add a table of the functions in <filename
+     class="headerfile">src/Field/FieldReductions.h</filename>.</para>
+ 
+     <para>FINISH: Add a table of the functions in <filename
+     class="headerfile">src/Field/DiffOps/FieldOffsetReductions.h</filename>.
+     QUESTION: Why is only <function>sum</function> defined?</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-expressions">
+     <title>Expressions Involving &field;s</title>
+ 
+     <para>Do something similar to &array;'s section.  See the
+     operations defined in <filename
+     class="headerfile">src/Field/FieldOperators.h</filename>,
+     <filename
+     class="headerfile">src/Field/FieldOperatorSpecializations.h</filename>,
+     <filename
+     class="headerfile">src/Field/PoomaFieldOperators.h</filename>, and
+     <filename
+     class="headerfile">src/Field/VectorFieldOperators.h</filename>.</para>
+ 
+     <para>Some operations involve both &array; and &field;
+     parameters.  Where do we list them?</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-stencils">
+     <title>&field; Stencils: Faster, Local Computations</title>
+ 
+     <para>ADD: a description of a stencil.  Why is it needed?  How
+     does a user use it?  How does a user know when to use one?  Add
+     documentation of the material from <filename
+     class="headerfile">src/Field/DiffOps/FieldStencil.h</filename>.</para>
+ 
+     <para>How is <type>FieldShiftEngine</type> used by &field;
+     stencils?  Should it be described here or in the &engine; section?
+     See the the code in <filename
+     class="headerfile">src/Field/DiffOps/FieldShiftEngine.h</filename>.</para>
+    </section>
+ 
+ 
+    <section>
+     <title>Cell Volumes, Face Areas, Edge Lengths, Normals</title>
+ 
+     <para>ADD: a description of these functions.  See <filename
+     class="headerfile">src/Field/Mesh/MeshFunctions.h</filename>.
+     These are initialized in, e.g., <filename
+     class="headerfile">src/Field/Mesh/UniformRectilinearMesh.h</filename>.
+     Note that these do not work for <type>NoMesh</type>.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-div">
+     <title>Divergence Operators</title>
+ 
+     <para>ADD: a table having divergence operators, explaining the
+     current restrictions imposed by what is implemented.  See
+     <filename class="headerfile">src/Field/DiffOps/Div.h</filename>
+     and <filename
+     class="headerfile">src/Field/DiffOps/Div.UR.h</filename>.  What
+     restrictions does <quote>UR</quote> (mesh) connote?</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-utilities">
+     <title>Utility Functions</title>
+ 
+     <section id="fields_ref-utilities-compression">
+      <title>Compressed Data</title>
+ 
+      <para>Add a table containing
+      <function>elementsCompressed</function>,
+      <function>compressed</function>, <function>compress</function>,
+      and <function>uncompress</function>.</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-utilities-sizes">
+      <title>Centering Sizes and Number of Materials</title>
+ 
+      <para>ADD: a description of <function>numMaterials</function> and
+      <function>centeringSize</function> found in <filename
+      class="headerfile">src/Field/Field.h</filename>.</para>
+ 
+      <para>QUESTION: How do these relate to any method functions?</para>
+     </section>
+ 
+ 
+     <section id="fields_ref-utilities-subfield">
+      <title>Obtaining Subfields</title>
+ 
+      <para>ADD: a description of <function>subField</function> found
+      in <filename
+      class="headerfile">src/Field/Field.h</filename>.</para>
+     </section>
+ 
+    </section>
+ 
+ 
+    <section id="fields_ref-centerings">
+     <title>&field; Centerings</title>
+ 
+     <para>DO: Describe the purpose of a centering and its definition.
+     Describe the ability to obtain canonical centerings.  Explain how
+     to construct a unique centering.  See <filename
+     class="headerfile">src/Field/FieldDentering.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-fieldoffsets">
+     <title>Relative &field; Positions</title>
+ 
+     <para>Permit specifying field positions relative to a field
+     location.  Describe <type>FieldOffset</type> and
+     <type>FieldOffsetList</type>.  See <filename
+     class="headerfile">src/Field/FieldOffset.h</filename></para>
+    </section>
+ 
+ 
+    <section id="fields_ref-nearestneighbors">
+     <title>Computing Close-by Field Positions</title>
+ 
+     <para>Given a field location, return the set of field locations
+     that are closest using ?Manhattan? distance.  See <filename
+     class="headerfile">src/Field/NearestNeighbors.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="fields_ref-mesh">
+     <title>Mesh ???</title>
+ 
+     <para>Unlike &array;s, &field;s are distributed throughout space
+     so distances between values within the &field can be computed.  A
+     &field;'s mesh stores this spatial distribution.</para>
+ 
+     <para>QUESTION: What do we need to write about meshes?  What is
+     unimportant implementation and what should be described in this
+     reference section?</para>
+ 
+     <para>QUESTION: Where in here should emphasize vertex, not cell,
+     positions?  <quote>VERTEX</quote> appears repeatedly in <filename
+     class="headerfile">src/Field/Mesh/NoMesh.h</filename>.</para>
+ 
+     <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	   orient="port" pgwide="0">
+      <title>Mesh Types</title>
+ 
+      <tgroup cols="2" align="left">
+       <thead>
+        <row>
+ 	<entry>Mesh Type</entry>
+ 	<entry>Description</entry>
+        </row>
+       </thead>
+       <tbody>
+        <row>
+ 	<entry><type>NoMesh<Dim></type></entry>
+ 	<entry><para>no physical spacing, causing a &field; to mimic
+ 	an &array; with multiple engines.</para></entry>
+        </row>
+        <row>
+ 	<entry><type>UniformRectilinearMesh<Dim,T></type></entry>
+ 	<entry><para>physical spacing formed by the Cartesian product
+ 	of ????.</para></entry>
+        </row>
+       </tbody>
+      </tgroup>
+     </table>
+ 
+ 
+     <section id="fields_ref-mesh-accessors">
+      <title>Mesh Accessors</title>
+ 
+      <para>ADD: a table listing accessors, explaining the difference
+      between (physical and total) and (cell and vertex) domains.  See
+      <filename class="headerfile">src/Field/Mesh/NoMesh.h</filename>.
+      Also, include <function>spacings</function> and
+      <function>origin</function> in <filename
+      class="headerfile">src/Field/Mesh/UniformRectilinearMesh.h</filename>.
+      Note <type>NoMesh</type> does not provide the latter two.</para>
+     </section>
+ 
+    </section>
+ 
+ 
+    <section id="fields_ref-tmp">
+     <title>TMP: What do we do with these …? Remove this
+     section.</title>
+ 
+     <para>QUESTION: Do we describe the <type>Patch</type>
+     specialization for &field; at this place or in some common place?
+     Follow &array;'s lead.</para>
+ 
+     <para>QUESTION: Where do we describe <type>CreateLeaf</type> and
+     <type>MakeFieldReturn</type> in <filename
+     class="headerfile">src/Field/FieldCreateLeaf.h</filename> and
+     <filename
+     class="headerfile">src/Field/FieldMakeReturn.h</filename>.</para>
+ 
+     <para>QUESTION: What do we do with <type>FieldEnginePatch</type>
+     in <filename
+     class="headerfile">src/Field/FieldEngine/FieldEnginePatch.h</filename>.</para>
+    </section>
+   </chapter>
+ 
+ 
+   <chapter id="engines_ref">
+    <title>&engine;s</title>
+ 
+    <para>From a user's point of view, a container makes data available
+    for reading and writing.  In fact, the container's &engine; stores
+    the data or, if the data is computed, performs a computation to
+    yield the data.</para>
+ 
+    <para>FINISH: Introduce the various types of engines.  Add a table
+    with a short description of each engine type.</para>
+ 
+    <para>FINISH: First, we specify a generic &engine;'s interface.
+    Then, we present &engine; specializations.</para>
+    
+    <table frame="none" colsep="0" rowsep="0" tocentry="1"
+ 	  orient="port" pgwide="0">
+     <title>Types of &engine;s</title>
+ 
+     <tgroup cols="3" align="left">
+      <thead>
+       <row>
+        <entry>Engine Type</entry>
+        <entry>Engine Tag</entry>
+        <entry>Description</entry>
+       </row>
+      </thead>
+      <tbody>
+       <row>
+        <entry><para>Brick</para></entry>
+        <entry><para><literal>Brick</literal></para></entry>
+        <entry><para>Explicitly store all elements in, e.g., a &cc;
+ 	array.</para></entry>
+       </row>
+       <row>
+        <entry><para>Compressible</para></entry>
+        <entry><para><literal>CompressibleBrick</literal></para></entry>
+        <entry><para>If all values are the same, use constant storage
+ 	for that single value.  Otherwise, explicitly store all
+ 	elements.</para></entry>
+       </row>
+       <row>
+        <entry><para>Constant</para></entry>
+        <entry><para><literal>ConstantFunction</literal></para></entry>
+        <entry><para>Returns the same constant value for all
+        indices.</para></entry>
+       </row>
+       <row>
+        <entry><para>Dynamic</para></entry>
+        <entry><para><literal>Dynamic</literal></para></entry>
+        <entry><para>Manages a contiguous, local, one-dimensional,
+        dynamically resizable block of data.</para></entry>
+       </row>
+       <row>
+        <entry><para>Component Forwarding</para></entry>
+        <entry><para><literal>CompFwd<EngineTag,
+        Components></literal></para></entry>
+        <entry><para>Returns the specified components from
+        <literal>EngineTag</literal>'s engine.  Components are
+        <quote>pieces</quote> of multi-value elements such as vectors
+        and tensors.</para></entry>
+       </row>
+       <row>
+        <entry><para>Expression</para></entry>
+        <entry><para><literal>ExpressionTag<Expr></literal></para></entry>
+        <entry><para>Returns the value of the specified &pete;
+        expression.</para></entry>
+       </row>
+       <row>
+        <entry><para>Index Function</para></entry>
+        <entry><para><literal>IndexFunction<Functor></literal></para></entry>
+        <entry><para>Makes the function
+        <literal>Functor</literal>accepting indices mimic an
+        array.</para></entry>
+       </row>
+       <row>
+        <entry><para>MultiPatch</para></entry>
+        <entry><para><literal>MultiPatch<LayoutTag,PatchTag></literal></para></entry>
+        <entry><para>Support distributed computation using several
+        processors (???contexts???).  <literal>LayoutTag</literal>
+        indicates how the entire array is distributed among the
+        processors.  Each processor uses a <literal>PatchTag</literal>
+        engine.</para></entry>
+       </row>
+       <row>
+        <entry><para>Remote</para></entry>
+        <entry><para><literal>Remote<EngineTag></literal></para></entry>
+        <entry><para>unknown</para></entry>
+       </row>
+       <row>
+        <entry><para>Remote Dynamic</para></entry>
+        <entry><para><literal>Remote<Dynamic></literal></para></entry>
+        <entry><para>unknown: specialization</para></entry>
+       </row>
+       <row>
+        <entry><para>Stencil</para></entry>
+        <entry><para><literal>StencilEngine<Function,
+        Expression></literal></para></entry>
+        <entry><para>Returns values computed by applying the
+        user-specified function to sets of contiguous values in the
+        given engine or container.  Compare with user function
+        engines.</para></entry>
+       </row>
+       <row>
+        <entry><para>User Function</para></entry>
+        <entry><para><literal>UserFunctionEngine<UserFunction,Expression></literal></para></entry>
+        <entry><para> Returns values computed by applying the
+        user-specified function to the given engine or container.
+        QUESTION: Is the following claim correct? For each returned
+        value, only one value from the engine or container is
+        used.</para></entry>
+       </row>
+      </tbody>
+     </tgroup>
+    </table>
+ 
+    <para>QUESTION: Where do we describe views?</para>
+ 
+    <para>QUESTION: What does <type>NewEngine</type> do?  Should it be
+     described when describing views?  Should it be omitted as an
+     implementation detail?</para>
+ 
+    <para>QUESTION: Where do we describe &engine; patches found in
+    <filename class="headerfile">src/Engine/EnginePatch.h</filename>?
+    All patch data in a separate chapter or engine-specific pieces in
+    this chapter?</para>
+ 
+    <para>QUESTION: What is <function>notifyEngineWrite</function>?
+    See also <filename
+ 		      class="headerfile">src/Engine/NotifyEngineWrite.h</filename>.</para>
+ 
+    <para>QUESTION: What aspect of MultiPatch uses <type>IsValid</type> in
+    <filename
+ 	     class="headerfile">src/Engine/IsValidLocation.h</filename>?</para>
+ 
+    <para>QUESTION: Who uses intersections?  Where should this be
+    described?  See <filename
+ 			     class="headerfile">src/Engine/Intersector.h</filename>, <filename
+ 											       class="headerfile">src/Engine/IntersectEngine.h</filename>, and
+    <filename
+ 	     class="headerfile">src/Engine/ViewEngine.h</filename>.</para>
+ 
+    <section id="engines_ref-compile_time_interface">
+     <title>&engine; Compile-Time Interface</title>
+ 
+     <para>ADD: a table of template parameters ala &array;.  ADD:
+     compile-time types and values.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-constructors">
+     <title>Constructors and Destructors</title>
+ 
+     <para>ADD: a table of constructors and destructors ala
+     &array;'s.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-access">
+     <title>Element Access</title>
+ 
+     <para>ADD: a table with <methodname>read</methodname> and
+     <methodname>operator()</methodname>.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-accessors">
+     <title>Accessors</title>
+ 
+     <para>ADD: a table of accessors.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-assignments">
+     <title>&engine; Assignments</title>
+ 
+     <para>similar to &array;'s assignments.  shallow copies.  ADD: a
+     table with one entry</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-utilities">
+     <title>Utility Methods</title>
+ 
+     <para>ADD: a table including
+     <methodname>makeOwnCopy</methodname>.</para>
+ 
+     <para>QUESTION: What are <methodname>dataObject</methodname>,
+     <methodname>isShared</methodname>, and related methods?</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-implementation">
+     <title>Implementation Details</title>
+ 
+     <para>ADD: this section.  Explain that
+     <varname>dataBlock_m</varname> and <varname>data_m</varname> point
+     to the same place.  The latter speeds access, but what is the
+     purpose of the former?</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-brick">
+     <title>Brick and BrickView Engines</title>
+ 
+     <para>ADD: description of what a brick means.  ADD: whatever
+     specializations the class has, e.g.,
+     <methodname>offset</methodname>.</para>
+ 
+     <para>QUESTION: What does <type>DoubleSliceHelper</type> do?</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-compressible">
+     <title>Compressible Brick and BrickView Engines</title>
+ 
+     <para>ADD this.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-dynamic">
+     <title>Dynamic and DynamicView Engines:</title>
+ 
+     <para>ADD this.  Manages a contiguous, local, resizable, 1D block
+     of data.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-components">
+     <title>Component Engines</title>
+ 
+     <para>I believe these implement array component-forwarding.  See
+     <filename
+ 	      class="headerfile">src/Engine/ForwardingEngine.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-expressions">
+     <title>Expression Engines</title>
+ 
+     <para>Should this be described in the &pete; section?  Unlikely.
+     See <filename
+ 		  class="headerfile">src/Engine/ExpressionEngine.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-enginefunctor">
+     <title>&engine; Functors</title>
+ 
+     <para>QUESTION: What is an <type>EngineFunctor</type>?  Should it
+     have its own section?  See <filename
+ 					 class="headerfile">src/Engine/EngineFunctor.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="engines_ref-fieldengines">
+     <title><type>FieldEngine</type>: A Hierarchy of &engine;s</title>
+ 
+     <para>A &field; consists of a hierarchy of materials and
+     centerings.  These are implemented using a hierarchy of engines.
+     See <filename
+ 		  class="headerfile">src/Field/FieldEngine/FieldEngine.h</filename>
+     and <filename
+ 		  class="headerfile">src/Field/FieldEngine/FieldEngine.ExprEngine.h</filename>.</para>
+    </section>
+   </chapter>
+ 
+ 
+   <chapter id="benchmarks_ref">
+    <title>&benchmark; Programs</title>
+ 
+    <para>Explain how to use &benchmark; programs, especially the
+    options.  Explain how to write a &benchmark; program.  See also
+    <filename class="headerfile">src/Utilities/Benchmark.h</filename>
+    and <filename
+    class="libraryfile">src/Utilities/Benchmark.cmpl.cpp</filename>.</para>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="layouts_ref">
+    <title>Layouts and Partitions: Distribute Computation Among
+    Contexts</title>
+ 
+    <para>QUESTION: What is the difference between
+    <type>ReplicatedTag</type> and <type>DistributedTag</type>?</para>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="pete_ref">
+    <title>&pete;: Evaluating Parallel Expressions</title>
+ 
+    <section>
+     <title>UNKNOWN</title>
+ 
+     <section id="pete_ref-unknown-leaf-tags">
+      <title>Leaf Tag Classes</title>
+ 
+      <para><type>NotifyPreReadTag</type> indicates a term is about to
+      be read.  Why is this needed?  Defined in <filename
+      class="headerfile">src/Utilities/NotifyPreRead.h</filename>.</para>
+     </section>
+    </section>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="views_ref">
+    <title>Views</title>
+ 
+    <para>QUESTION: Should this have its own chapter or be part of a
+    container chapter?</para>
+ 
+    <para>Describe <type>View0</type>, <type>View1</type>, …,
+    <type>View7</type> and <type>View1Implementation</type>.</para>
+ 
+    <para>QUESTION: What causes the need for <type>AltView0</type> and
+    <type>AltComponentView</type>?</para>
+ 
+    <para>Be sure to describe <type>ComponentView</type> in the same
+    place.  This is specialized for &array;s in <filename
+    class="headerfile">src/Array/Array.h</filename>:1323–1382.</para>
+ 
+    <section>
+     <title><type>ViewIndexer<Dim,Dim2></type></title>
+ 
+     <para>Defined in <filename
+     class="headerfile">src/Utilities/ViewIndexer.h</filename>, this
+     type translates indices between a domain and a view of it.</para>
+    </section>
+   </chapter>
+ 
+   <chapter id="threads_ref">
+    <title>Threads</title>
+ 
+    <para>Perhaps include information in <filename
+    class="headerfile">src/Engine/DataObject.h</filename>.</para>
+ 
+    <para>&pooma; options include UNFINISHED</para>
+ 
+   </chapter>
+ 
+ 
+   <chapter id="utilities_ref">
+    <title>Utility Types</title>
+ 
+    <para>TMP: What is a good order?</para>
+ 
+    <section id="utilities_ref-options">
+     <title><type>Options</type>: Varying Run-Time Execution</title>
+ 
+     <para>Each &pooma; executable has a <type>Options</type> object,
+     created by <function>Pooma::initialize</function>, storing
+     run-time configurable values found in <varname>argv</varname>.
+     Default options are found in
+     <methodname>Options::usage</methodname>.</para>
+ 
+     <para>See <filename
+     class="headerfile">src/Utilities/Options.h</filename> and
+     <filename
+     class="libraryfile">src/Utilities/Options.cmpl.cpp</filename>.</para>
+ 
+     <para>Scatter the specific options to other parts of the
+     manual.</para>
+    </section>
+ 
+    <section id="utilities_ref-assertions">
+     <title>Check Correctness: <type>CTAssert</type>,
+     <type>PAssert</type>, <type>PInsist</type>,
+     <type>SameType</type></title>
+ 
+     <para>Assertions ensure program invariants are obeyed.
+     <type>CTAssert</type>, checked at compile time, incur no run-time
+     cost.  <type>PAssert</type> and <type>PInsist</type> are checked
+     to run-time, the latter producing an explanatory message if the
+     assertion fails.  Compiling with <envar>NOCTAssert</envar> and
+     <envar>NOPTAssert</envar> disable these checks.  Compiling with just
+     <envar>NOPTAssert</envar> disables only the run-time checks.</para>
+ 
+     <para><type>SameType</type> ensures, at compile-time, two types
+     are the same.</para>
+ 
+     <para>These are implemented in <filename
+     class="headerfile">src/Utilities/PAssert.h</filename> and
+     <filename
+     class="libraryfile">src/Utilities/PAssert.cmpl.cpp</filename>.</para>
+    </section>
+ 
+    <section id="utilities_ref-clock">
+     <title><type>Clock</type>: Measuring a Program's Execution Time</title>
+ 
+     <para>See <filename
+     class="headerfile">src/Utilities/Clock.h</filename>.</para>
+    </section>
+ 
+ 
+    <section id="utilities_ref-smart_pointers">
+     <title>Smart Pointers: <type>RefCountedPtr</type>,
+     <type>RefCountedBlockPtr</type>, and
+     <type>DataBlockPtr</type></title>
+ 
+     <para>See <filename
+     class="headerfile">src/Utilities/{RefCountedPtr,RefCountedBlockPtr,DataBlockPtr}.h</filename>.
+     <filename class="headerfile">src/Utilities/RefCounted.h</filename>
+     helps implement it.  <type>DataBlockPtr</type> uses
+     &smarts;.</para>
+    </section>
+ 
+    <section id="utilities_ref-inform">
+     <title><type>Inform</type>: Formatted Output for Multi-context
+     Execution</title>
+ 
+     <para>See <filename
+     class="headerfile">src/Utilities/Inform.h</filename> and <filename
+     class="libraryfile">src/Utilities/Inform.cmpl.cpp</filename>.</para>
+    </section>
+ 
+    <section id="utilities_ref-statistics">
+     <title><type>Statistics</type>: Report &pooma; Execution Statistics</title>
+ 
+     <para>Collect and print execution statistics.  Defined in
+     <filename
+     class="headerfile">src/Utilities/Statistics.h</filename>.</para>
+    </section>
+ 
+    <section id="utilities_ref-random_numbers">
+     <title>Random Numbers: <type>Unique</type></title>
+ 
+     <para>See <filename
+     class="headerfile">src/Utilities/Unique.h</filename>.</para>
+    </section>
+   </chapter>
+ 
+ 
+   <chapter id="implementation_types_ref">
+    <title>Types for Implementing &pooma;</title>
+ 
+    <para>TMP: What is a good order?</para>
+ 
+    <para>Describe types defined to implement &pooma; but that users do
+    not directly use.  This chapter has lower priority than other
+    chapters since users (hopefully) do not need to know about these
+    classes.</para>
+ 
+    <section id="implementation_types_ref-tester">
+     <title><type>Tester</type>: Check Implementation Correctness</title>
+ 
+     <para>&pooma; implementation test programs frequently consist of a
+     series of operations followed by correctness checks.  The
+     <type>Tester</type> object supports these tests, returning a
+     boolean whether all the correctness checks yield true.  Under
+     verbose output, messages are printed for each test.  See <filename
+     class="headerfile">src/Utilities/Tester.h</filename>.</para>
+    </section>
+ 
+    <section id="implementation_types_ref-elementproperties">
+     <title><type>ElementProperties<T></type>: Properties a Type
+     Supports</title>
+ 
+     <para>This traits class permits optimizations in other templated
+     classes.  See <filename
+     class="headerfile">src/Utilities/ElementProperties.h</filename>.</para>
+ 
+    </section>
+ 
+    <section id="implementation_types_ref-typeinfo">
+     <title><type>TypeInfo<T></type>: Print a String Describing
+     the Type</title>
+ 
+     <para>Print a string describing the type.  Defined in <filename
+     class="headerfile">src/Utilities/TypeInfo.h</filename>.  It is
+     specialized for other types in other files, e.g., <filename
+     class="headerfile">src/Engine/EngineTypeInfo.h</filename> and
+     <filename class="headerfile">src/Field/FieldTypeInfo.h</filename>.
+     Is this a compile-time version of RTTI?</para>
+    </section>
+ 
+    <section id="implementation_types_ref-looputils">
+     <title><type>LoopUtils</type>: Loop Computations at Compile Time</title>
+ 
+     <para>At compile time, <type>LoopUtils</type> supports copying
+     between arrays and computing the dot product of arrays.  See
+     <filename
+ 	      class="headerfile">src/Utilities/MetaProg.h</filename>.</para>
+    </section>
+ 
+    <section id="implementation_types_ref-modelelement">
+     <title><type>ModelElement<T></type>: Wrap a Type</title>
+ 
+     <para>A wrapper class used to differentiate overloaded functions.
+     Defined in <filename
+     class="headerfile">src/Utilities/ModelElement.h</filename>.  Used
+     only by &array; and <type>DynamicArray</type>.</para>
+    </section>
+ 
+    <section id="implementation_types_ref-wrappedint">
+     <title><type>WrappedInt<int></type>: Wrap a Number</title>
+ 
+     <para>A wrapper class used to differentiate overloaded functions
+     among different integers.  Defined in <filename
+     class="headerfile">src/Utilities/WrappedInt.h</filename>.  Is this
+     class deprecated?  Is it even necessary?</para>
+    </section>
+    
+    <section id="implementation_types_ref-empty_classes">
+     <title>Supporting Empty Classes</title>
+ 
+     <para>The <type>NoInit</type> tag class indicates certain
+     initializations should be skipped.  Defined in <filename
+     class="headerfile">src/Utilities/NoInit.h</filename>.</para>
+ 
+     <para>FIXME: Should be macro, not function.
+     <function>POOMA_PURIFY_CONSTRUCTORS</function> generates an empty
+     constructor, copy constructor, and destructor to avoid &purify;
+     warnings.  Defined in <filename
+     class="headerfile">src/Utilities/PurifyConstructors.h</filename>.</para>
+ 
+    </section>
+ 
+    <section id="implementation_types_ref-pool">
+     <title><type>Pooled<T></type>: Fast Memory Allocation of
+     Small Blocks</title>
+ 
+     <para><type>Pooled<T></type> speeds allocation and
+     deallocation of memory blocks for small objects with
+     type <type>T</type>.  Defined in <filename
+     class="headerfile">src/Utilities/Pooled.h</filename>, it is
+     implemented in <filename
+     class="headerfile">src/Utilities/Pool.h</filename> and <filename
+     class="libraryfile">src/Utilities/Pool.cmpl.cpp</filename>.
+     <filename class="headerfile">src/Utilities/StaticPool.h</filename>
+     no longer seems to be used.</para>
+    </section>
+ 
+    <section id="implementation_types_ref-uninitialized_vector">
+     <title><type>UninitializedVector<T,Dim></type>: Create
+     Without Initializing</title>
+ 
+     <para>This class optimizes creation of an array of objects by
+     avoiding running the default constructors.  Later initialization
+     can occur, perhaps using a loop that can be unrolled.  Defined in
+     <filename
+     class="headerfile">src/Utilities/UninitializedVector.h</filename>,
+     this is used only by <type>DomainTraits</type>.</para>
+    </section>
+   </chapter>
+ 
+   <chapter id="implementation_algorithms_ref">
+    <title>Algorithms for Implementing &pooma;</title>
+ 
+    <para>In <filename>src/Utilities/algorithms.h</filename>,
+    <function>copy</function>, <function>delete_back</function>, and
+    <function>delete_shiftup</function> provide additional algorithms
+    using iterators.</para>
+   </chapter>
+ 
+ 
+   <chapter id="where-place-these_ref">
+    <title>TMP: Where do we describe these files?</title>
+ 
+    <itemizedlist>
+     <listitem>
+      <para><filename
+ 		     class="headerfile">src/Utilities/Conform.h</filename>: tag for
+      checking whether terms in expression have conforming
+      domains</para>
+     </listitem>
+ 
+     <listitem>
+      <para><filename
+ 		     class="headerfile">src/Utilities/DerefIterator.h</filename>:
+      <type>DerefIterator<T></type> and
+      <type>ConstDerefIterator<T></type> automatically
+      dereference themselves to maintain <literal>const</literal>
+      correctness.  <!-- FIXME: s/literal/keyword/ --></para>
+     </listitem>
+ 
+     <listitem>
+      <para><filename
+ 		     class="headerfile">src/Utilities/Observable.h</filename>,
+      <filename class="headerfile">src/Utilities/Observer.h</filename>,
+      and <filename
+ 		   class="headerfile">src/Utilities/ObserverEvent.h</filename>:
+      <type>Observable<T></type>,
+      <type>SingleObserveable<T></type>,
+      <type>Observer<T></type>, and <type>ObserverEvent</type>
+      implement the observer pattern.  What is the observer pattern?
+      Where is this used in the code?</para>
+     </listitem>
+    </itemizedlist>
+ 
+   </chapter>
+ 
+  </part>
+ 
+ 
+  <appendix id="future_development">
+   <title>Future Development</title>
+ 
+   <section id="future_development-particles">
+    <title>Particles</title>
+ 
+    <para><filename
+ 		   class="libraryfile">docs/ParticlesDoc.txt</filename> has
+    out-of-date information.</para>
+ 
+    <para>See Section 3.2.3 of
+    <filename>papers/pooma.ps</filename> for an out-of-date
+    description.</para>
+ 
+    <para><filename>papers/jvwr.ps</filename> concerns mainly
+    particles.  <filename>papers/8thSIAMPOOMAParticles.pdf</filename>,
+    by Julian Cummings and Bill Humphrey, concerns parallel particle
+    simulations.  <filename>papers/iscope98linac.pdf</filename>
+    describes a particle beam simulation using &pooma;; it mainly
+    concerns particles.</para>
+ 
+    <section>
+     <title>Particles</title>
+ 
+     <para>Do we want to include such a section?</para>
+ 
+     <para>Section 3, "Sample Applications" of
+     <filename>papers/SiamOO98_paper.ps</filename> describes porting a
+     particle program written using High-Performance Fortran to
+     &pooma; and presumably why particles were added to &pooma;.  It
+     also describes <application>MC++</application>, a Monte Carlo
+     neutron transport code.</para>
+ 
+    </section>
+ 
+   </section>
+ 
+ 
+   <section id="future_development-composition_engine">
+    <title>Composition of &engine;s</title>
+ 
+    <para>The i,j-th element of the composition
+    <varname>a</varname>∘<varname>b</varname> of two arrays
+    <varname>a</varname> and <varname>b</varname> equals a(b(i,j)).
+    The composition engine tagged <literal>IndirectionTag<Array1,
+    Array2></literal>, defined in <filename
+ 					      class="headerfile">src/Engine/IndirectionEngine.h</filename> is
+    unfinished.</para>
+   </section>
+ 
+ 
+   <section id="future_development-container_consistency">
+    <title>Improving Consistency of Container Interfaces</title>
+ 
+    <section id="future_development-container_consistency-array_relations">
+     <title>Relations for &array;s</title>
+ 
+     <para>Do &array;s currently support relations?  If not, why not?
+    Should they be added?</para>
+    </section>
+ 
+    <section id="future_development-container_consistency-dimensions">
+     <title>Supporting the Same Number of Dimensions</title>
+ 
+     <para>&array; and &field; should support the same maximum number
+     of dimensions.  Currently, &array;s support seven dimensions and
+     &field;s support only three.  By definition, &dynamicarray;
+     supports only one dimension.</para>
+ 
+     <para>Relations for &array;s.</para>
+ 
+     <para>External guards for &array;s.</para>
+    </section>
+ 
+   </section>
+ 
+ 
+   <section id="future_development-where">
+    <title><function>where</function> Proxies</title>
+ 
+    <para>QUESTION: Do we even discuss this broken
+    <quote>feature</quote>?  Where is it used?  Some related code is in
+    <filename>src/Array/Array.h</filename>:2511–2520.</para>
+   </section>
+ 
+ 
+   <section id="future_development-long_term">
+    <title>Very Long Term Development Ideas</title>
+ 
+    <para>Describe how to write a new configuration file.</para>
+   </section>
+ 
+  </appendix>
+ 
+ 
+  <appendix id="installation">
+   <title>Obtaining and Installing &pooma;</title>
+ 
+   <para>ADD: Write this section, including extensive instructions
+   for Unix, MS Windows, and MacOS.  List the configuration options.
+   Be sure to describe configuring for parallel execution.</para>
+ 
+   <section id="installation-distributed_computing">
+    <title>Supporting Distributed Computation</title>
+ 
+    <para>To use multiple processors with &pooma; requires installing
+    the &cheetah; messaging library and an underlying messaging library
+    such as the Message Passing Interface (&mpi;) Communications
+    Library or the &mm; Shared Memory Library.  In this section, we
+    first describe how to install &mm;.  Read the section only if using
+    &mm;, not &mpi;.  Then we describe how to install &cheetah; and
+    configure &pooma; to use it.</para>
+ 
+    <section id="installation-distributed_computing-mm">
+     <title>Obtaining and Installing the &mm; Shared Memory Library</title>
+ 
+     <para>&cheetah;, and thus &pooma;, can use Ralf Engelschall's &mm;
+     Shared Memory Library to pass messages between processors.  For
+     example, the &author; uses this library on a two-processor
+     computer running &linux;.  The library, available at
+     http://www.engelschall.com/sw/mm/, is available for free and has
+     been successfully tested on a variety of Unix platforms.</para>
+ 
+     <para>We describe how to download and install the &mm; library.
+      <orderedlist spacing="compact">
+ 	<listitem>
+ 	 <para>Download the library from the &pooma; Download page
+          available off the &pooma; home page (&poomaHomePage;).</para>
+ 	</listitem>
+ 	<listitem>
+ 	 <para>Extract the source code using <command>tar xzvf
+          mm-1.1.3.tar.gz</command>.  Move into the resulting source
+          code directory <filename
+          class="directory">mm-1.1.3</filename>.</para>
+ 	</listitem>
+ 	<listitem>
+ 	 <para>Prepare to compile the source code by configuring it
+          using the <command>configure</command> command.  To change
+          the default installation directory <filename
+          class="directory">/usr/local</filename>, specify
+          <command>--prefix=<replaceable>directory</replaceable></command>
+          option.  The other configuration options can be listed by
+          specifying the <command>--help</command> option.  Since the
+          &author; prefers to keep all &pooma;-related code in his
+          <filename class="directory">pooma</filename>subdirectory, he
+          uses <command>./configure
+          --prefix=${HOME}/pooma/mm-1.1.3</command>.</para>
+ 	</listitem>
+ 	<listitem>
+ 	 <para>Create the library by issuing the <command>make</command>
+      command.  This compiles the source code using a &c; compiler.  To
+      use a different compiler than the &mm; configuration chooses, set
+      the <envar>CC</envar> to the compiler before configuring.</para>
+       </listitem>
+     <listitem>
+      <para>Optionally test the library by issuing the <command>make
+      test</command> command.  If successful, the penultimate line
+      should be <computeroutput>OK - ALL TESTS SUCCESSFULLY
+      PASSED</computeroutput>.</para>
+     </listitem>
+     <listitem>
+      <para>Install the &mm; Library by issuing the <command>make
+      install</command> command.  This copies the library files to the
+      installation directory.  The <filename
+      class="directory">mm-1.1.3</filename> directory containing the
+      source code may now be removed.</para>
+     </listitem>
+    </orderedlist>
+    </para>
+    </section>
+ 
+    
+    <section id="installation-distributed_computing-cheetah">
+     <title>Obtaining and Installing the &cheetah; Messaging Library</title>
+ 
+     <para>The &cheetah; Library decouples communication from
+     synchronization.  Using asynchronous messaging rather than
+     synchronous messaging permits a message sender to operate without
+     the cooperation of the message recipient.  Thus, implementing
+     message sending is simpler and processing is more efficiently
+     overlapped with it.  Remote method invocation is also supported.
+     The library was developed at the Los Alamos National Laboratory's
+     Advanced Computing Laboratory.</para>
+ 
+     <para>&cheetah;'s messaging is implemented using an underlying
+     messaging library such as the Message Passing Interface (&mpi;)
+     Communications Library (FIXME: xref linkend="mpi99", <ulink
+     url="http://www-unix.mcs.anl.gov/mpi/"></ulink>) or the &mm;
+     Shared Memory Library.  &mpi; works on a wide variety of platforms
+     and has achieved widespread usage.  &mm; works under Unix on any
+     computer with shared memory.  Both libraries are available for
+     free.  The instructions below work for whichever library you
+     choose.</para>
+ 
+     <para>We describe how to download and install &cheetah;.
+      <orderedlist spacing="compact">
+       <listitem>
+        <para>Download the library from the &pooma; Download page
+        available off the &pooma; home page (&poomaHomePage;).</para>
+       </listitem>
+       <listitem>
+        <para>Extract the source code using <command>tar xzvf
+        cheetah-1.0.tgz</command>.  Move into the resulting source code
+        directory <filename
+        class="directory">cheetah-1.0</filename>.</para>
+       </listitem>
+       <listitem>
+        <para>Edit a configuration file corresponding to your operating
+        system and compiler.  These <filename
+        class="libraryfile">.conf</filename> files are located in the
+        <filename class="directory">config</filename> directory.  For
+        example, to use &gcc; with the &linux; operating system, use
+        <filename
+        class="libraryfile">config/LINUXGCC.conf</filename>.</para>
+ 
+        <para>The configuration file usually does not need
+        modification.  However, if you are using &mm;, ensure
+        <varname>shmem_default_dir</varname> specifies its location.
+        For example, the &author; modified the value to
+        <statement>"/home/oldham/pooma/mm-1.1.3"</statement>.</para>
+       </listitem>
+       <listitem>
+        <para>Prepare to compile the source code by configuring it
+        using the <command>configure</command> command.  Specify the
+        configuration file using the <command>--arch</command> option.
+        Its argument should be the configuration file's name, omitting
+        its <filename class="libraryfile">.conf</filename> suffix.  For
+        example, <command>--arch LINUXGCC</command>.  Some other
+        options include
+        <variablelist>
+        <varlistentry>
+ 	<term>--help</term>
+ 	<listitem>
+ 	 <para>lists all the available options</para>
+ 	</listitem>
+        </varlistentry>
+        <varlistentry>
+ 	<term>--shmem --nompi</term>
+ 	<listitem>
+ 	 <para>indicates use of &mm;, not &mpi;</para>
+ 	</listitem>
+        </varlistentry>
+        <varlistentry>
+ 	<term>--mpi --noshmem</term>
+ 	<listitem>
+ 	 <para>indicates use of &mpi;, not &mm;</para>
+ 	</listitem>
+        </varlistentry>
+        <varlistentry>
+ 	<term>--opt</term>
+ 	<listitem>
+ 	 <para>causes the compiler to produce optimized source code</para>
+ 	</listitem>
+        </varlistentry>
+        <varlistentry>
+ 	<term>--noex</term>
+ 	<listitem>
+ 	 <para>prevents use of &cc; exceptions</para>
+ 	</listitem>
+        </varlistentry>
+        <varlistentry>
+ 	<term>--prefix <replaceable>directory</replaceable></term>
+ 	<listitem>
+ 	 <para>specifies the installation directory where the
+ 	   library will be copied rather than the default.</para>
+ 	</listitem>
+        </varlistentry>
+       </variablelist>
+         For example, the &author; uses <command>./configure --arch
+         LINUXGCC --shmem --nompi --noex --prefix
+         ${HOME}/pooma/cheetah-1.0 --opt</command>.  The
+         <command>--arch LINUXGCC</command> indicates use of &gcc;
+         under a &linux; operating system.  The &mm; library is used,
+         but &cc; exceptions are not.  The latter choice matches
+         &pooma;'s default choice.  The library will be installed in
+         the <filename
+ 		      class="directory">${HOME}/pooma/cheetah-1.0</filename>.
+         Finally, the library code will be optimized, hopefully running
+         faster than unoptimized code.</para>
+       </listitem>
+       <listitem>
+        <para>Follow the directions printed by
+        <command>configure</command>: Change directories to the
+        <filename class="directory">lib</filename> subdirectory named
+        by the <command>--arch</command> argument and then type
+        <command>make</command> to compile the source code and create
+        the library.</para>
+       </listitem>
+       <listitem>
+        <para>Optionally ensure the library works correctly by issuing
+        the <command>make tests</command> command.</para>
+       </listitem>
+       <listitem>
+        <para>Install the library by issuing the <command>make
+        install</command> command.  This copies the library files to
+        the installation directory.  The <filename
+        class="directory">cheetah-1.0</filename> directory containing
+        the source code may now be removed.</para>
+       </listitem>
+      </orderedlist>
+    </para>
+    </section>
+ 
+    <section id="installation-distributed_computing-pooma">
+     <title>Configuring &pooma; When Using &cheetah;</title>
+ 
+     <para>To use &pooma; with &cheetah;, one must tell &pooma; the
+     location of the &cheetah; library using the
+     <command>--messaging</command> configuration option.  To do this,
+      <orderedlist spacing="compact">
+       <listitem>
+        <para>Set the &cheetah; directory environment variable
+         <envar>CHEETAHDIR</envar> to the directory containing the
+         installed &cheetah; library.  For
+         example, <command>declare -x
+         CHEETAHDIR=${HOME}/pooma/cheetah-1.0</command> specifies the
+         installation directory used in the previous section.</para>
+       </listitem>
+       <listitem>
+        <para>When configuring &pooma;, specify the
+        <command>--messaging</command> option.  For example,
+        <command>./configure --arch LINUXgcc --opt
+        --messaging</command> configures for &linux;, &gcc;, and an
+        optimized library using &cheetah;.</para>
+       </listitem>
+      </orderedlist>
+     </para>
+    </section>
+ 
+ <!-- HERE -->
+   </section>
+  </appendix>
+ 
+ 
+  <appendix id="compilation_errors">
+   <title>Dealing with Compilation Errors</title>
+ 
+   <para>Base this low-priority section on <filename
+   class="libraryfile">errors.html</filename>.  QUESTION: Where is
+   <filename class="libraryfile">errors.html</filename>?</para>
+ 
+  </appendix>
+ 
+ 
+  <appendix id="tmp_appendix">
+   <title>TMP: Notes to Myself</title>
+ 
+   <section id="tmp-miscellaneous">
+    <title>Miscellaneous</title>
+ 
+    <orderedlist>
+     <listitem>
+      <para>QUESTION: How do I know when to use a type name versus just
+      the concept?  For example, when do I use <quote>array</quote>
+      versus &array;?</para>
+     </listitem>
+ 
+     <listitem>
+      <para>Krylov solvers are described in Section 3.5.2 of
+      <filename>papers/pooma.ps</filename>.</para>
+     </listitem>
+ 
+     <listitem>
+      <para>Section 5, "The Polygon Overlay Problem," describes
+      porting an <acronym>ANSI</acronym> &c; program to &pooma;.</para>
+     </listitem>
+ 
+     <listitem>
+      <para>A good example book: <emphasis>STL Tutorial and Reference
+      Guide: &cc; Programming with the Standard Template
+      Library</emphasis>, second edition, by David R. Musser,
+      Gillmer J. Derge, and Atul Sanai, ISBN 0-201-37923-6,
+      QA76.73.C153.M87 2001.</para>
+     </listitem>
+ 
+     <listitem>
+      <para>One STL reference book listed functions in margin notes,
+      easing finding material.  Do this.</para>
+     </listitem>
+ 
+     <listitem>
+      <para>QUESTION: Does Berna Massingill at Trinity University have
+      any interest ior access to any parallel computers?</para>
+     </listitem>
+    </orderedlist>
+ 
+   </section>
+ 
+ 
+   <section id="tmp-html_tutorials">
+    <title>Existing HTML Tutorials</title>
+ 
+    <para>All these tutorials are out-of-date, but the ideas and text
+    may still be relevant.</para>
+ 
+    <variablelist>
+     <varlistentry><term><filename
+ 				  class="libraryfile">index.html</filename></term>
+      <listitem><para>list of all tutorials.  No useful
+     material.</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">introduction.html</filename></term>
+      <listitem><para>data-parallel Laplace solver using Jacobi
+     iteration ala <command>Doof2d</command></para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">background.html</filename></term>
+      <listitem><para>short, indirect introduction to &pete;; parallel
+     execution model; &cc;; templates; &stl;; expression
+     templates</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">tut-01.html</filename></term>
+      <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">Layout.html</filename></term>
+      <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">parallelism.html</filename></term>
+      <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+ 				  class="libraryfile">self-test.html</filename></term>
+      <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">threading.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-03.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-04.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-05.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-06.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-07.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-08.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-09.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-10.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-11.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-12.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+     <varlistentry><term><filename
+     class="libraryfile">tut-13.html</filename></term>
+     <listitem><para>UNFINISHED</para></listitem>
+     </varlistentry>
+    </variablelist>
+ 
+   </section>
+ 
+  </appendix>
+ 
+ 
+  <!-- Bibliography -->
+ 
+  <bibliography id="bibliography">
+   <title>Bibliography</title>
+ 
+   <para>FIXME: How do I process these entries?</para>
+ 
+   <biblioentry>
+    <abbrev>mpi99</abbrev>
+    <authorgroup>
+     <author>
+      <firstname>William</firstname><surname>Gropp</surname>
+     </author>
+     <author>
+      <firstname>Ewing</firstname><surname>Lusk</surname>
+     </author>
+     <author>
+      <firstname>Anthony</firstname><surname>Skjellum</surname>
+     </author>
+    </authorgroup>
+    <copyright>
+     <year>1999</year>
+     <holder>Massachusetts Institute of Technology</holder>
+    </copyright>
+    <isbn>0-262-57132-3</isbn>
+    <publisher>
+     <publishername>The MIT Press</publishername>
+     <address>Cambridge, MA</address>
+    </publisher>
+    <title>Using MPI</title>
+    <subtitle>Portable Parallel Programming with the Message-Passing Interface</subtitle>
+    <edition>second edition</edition>
+   </biblioentry>
+  </bibliography>
+ 
+ 
+  <!-- Glossary -->
+ 
+  <glossary id="glossary">
+   <title>Glossary</title>
+ 
+   <para>ADD: Make sure all entries are indexed and perhaps point back
+   to their first use.  WARNING: This is constructed by hand so it is
+   likely to be full of inconsistencies and errors.</para>
+ 
+   <glossdiv>
+    <title>S</title>
+ 
+    <glossentry id="glossary-suite_name">
+     <glossterm>Suite Name</glossterm>
+     <glossdef>
+      <para>An arbitrary string denoting a particular toolkit
+      configuration.  For example, the string
+      <quote>SUNKCC-debug</quote> might indicate a configuration for
+      the <trademark>Sun</trademark> <productname>Solaris</productname>
+      operating system and the &kcc; &cc; compiler with debugging
+      support.  By default, the suite name it is equal to the
+      configuration's architecture name.</para>
+     </glossdef>
+    </glossentry>
+   </glossdiv>
+  </glossary>
+  
+ 
+  <!-- Index -->
+ 
+   &genindex.sgm;
+ 
+ </book>
Index: docs/manual/figures/distributed.mp
===================================================================
RCS file: distributed.mp
diff -N distributed.mp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- distributed.mp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,195 ----
+ %% Oldham, Jeffrey D.
+ %% 2001Nov28
+ %% Pooma
+ 
+ %% Illustrations for Distributed Computing
+ 
+ %% Assumes TEX=latex.
+ 
+ input boxes;
+ 
+ verbatimtex
+ \documentclass[10pt]{article}
+ \begin{document}
+ etex
+ 
+ %% Parts of Distributed Computation
+ beginfig(101)
+   numeric unit; unit = 0.9cm;
+ 
+   %% Create the Container Storage Partition subfigure.
+   numeric arrayWidth; arrayWidth = 2; % as multiple of unit
+   numeric arrayHeight; arrayHeight = 4; % as multiple of unit
+   numeric guardWidth; guardWidth = 0.1; % as multiple of unit
+   numeric patchWidth; patchWidth = arrayWidth/3; % as multiple of unit
+   numeric patchHeight; patchHeight = arrayHeight/2; % as multiple of unit
+   numeric xPatchDistance; xPatchDistance = 0.9patchWidth; % as multiple of unit
+   numeric yPatchDistance; yPatchDistance = 0.5patchWidth; % as multiple of unit
+   numeric arrayPartitionDistance; arrayPartitionDistance = arrayWidth;
+ 				% distance between array and partition
+   numeric arrayLayoutDistance; arrayLayoutDistance = 0.5arrayHeight;
+ 				% distance between array and layout
+   numeric arrowLayoutDistance; arrowLayoutDistance = 0.5arrayLayoutDistance;
+ 				% distance between arrow and top of layout, not its label
+   numeric iota; iota = labeloffset;
+   numeric storageBoundaryWidth; storageBoundaryWidth = 1; % as multiple of unit
+ 				% gap between storage box and its internals
+   
+   % Create the Array.  Use box "a".
+   boxit.a();
+   a.ne - a.sw = unit*(arrayWidth,arrayHeight);
+ 
+   % Create the partition.  Use boxes "p[]".
+   for t = 0 upto 5:
+     boxit.p[t]();
+     p[t].ne - p[t].sw = unit*(1,1);
+   endfor;
+   for t = 0 upto 2:
+     p[t].sw=p[t+3].nw;
+     p[t].se=p[t+3].ne;
+     if t < 2:
+       p[t].ne = p[t+1].nw;
+       p[t+3].ne = p[t+4].nw;
+     fi
+   endfor;
+   boxit.pt(btex \begin{tabular}{c} + \\ external guard layers \\ + \\ internal guard layers \end{tabular} etex);
+   pt.n = p[4].s;
+ 
+   % Create the layout patches "l[]" and their guard layers "g[]".
+   for t = 0 upto 5:
+     boxit.l[t]();
+     boxit.g[t]();
+     l[t].ne - l[t].sw = unit*(patchWidth, patchHeight);
+     g[t].ne - l[t].ne = -(g[t].sw - l[t].sw) = unit*guardWidth*(1,1);
+   endfor
+   for t = 0 upto 2:
+     if t < 2:
+       g[t+1].nw - g[t].ne = unit*(xPatchDistance,0);
+     fi
+     g[t].sw - g[t+3].nw = unit*(0,yPatchDistance);
+   endfor;
+ 
+   % Create the storage equation boxes.
+   boxit.containerPlus(btex + etex);
+   boxit.containerArrow(btex $\Big\Downarrow$ etex);
+ 
+   % Position the storage pieces.
+   p[0].nw - a.ne = unit*(arrayPartitionDistance,0);
+   containerPlus.c = (xpart(0.5[a.ne,p[0].nw]), ypart(a.c));
+   containerArrow.c = (xpart(containerPlus.c), ypart(l[1].n) + unit*arrowLayoutDistance);
+   ypart(a.s - l[1].n) = unit*arrayLayoutDistance;
+   xpart(containerPlus.c - l[1].n) = 0;
+ 
+   % Create a boundary box around storage partition.
+   boxit.storageBoundary();
+   ypart(storageBoundary.n - a.n) =
+   ypart(l[4].s - storageBoundary.s) = unit*2storageBoundaryWidth;
+   xpart(a.w - storageBoundary.w) = unit*storageBoundaryWidth;
+   xpart(storageBoundary.e - pt.e) = unit*storageBoundaryWidth;
+   
+   %% Create the Computer Configuration subfigure. 
+   numeric configurationBoundaryWidth; configurationBoundaryWidth = storageBoundaryWidth;
+ 				% gap between computer configuration box and its internals
+ 
+   for t = 0 upto 2:
+     circleit.c[t]();
+     c[t].n - c[t].s = 1.3(0,ypart(g[0].ne - g[3].sw));
+     c[t].e - c[t].w = 1.5(xpart(g[0].ne - g[3].sw),0);
+   endfor
+   c[2].c - c[1].c = c[1].c - c[0].c = g[1].c - g[0].c;
+ 
+   boxit.configurationBoundary();
+   ypart(configurationBoundary.n - configurationBoundary.s) =
+   ypart(storageBoundary.n - storageBoundary.s);
+   xpart(configurationBoundary.e - configurationBoundary.w) =
+   xpart(c[2].e - c[0].w)+2*unit*configurationBoundaryWidth;
+   configurationBoundary.c = c[1].c;
+ 
+   %% Create the Computation Configuration subfigure.
+   % Create the patches.
+   for t = 0 upto 5:
+     boxit.L[t]();
+     boxit.G[t]();
+     L[t].ne - L[t].sw = unit*(patchWidth, patchHeight);
+     G[t].ne - L[t].ne = -(G[t].sw - L[t].sw) = unit*guardWidth*(1,1);
+   endfor
+   for t = 0 upto 2:
+     if t < 2:
+       G[t+1].nw - G[t].ne = unit*(xPatchDistance,0);
+     fi
+     G[t].sw - G[t+3].nw = unit*(0,yPatchDistance);
+   endfor;
+ 
+   % Create the contexts.
+   for t = 0 upto 2:
+     circleit.C[t]();
+     C[t].n - C[t].s = 1.3(0,ypart(G[0].ne - G[3].sw));
+     C[t].e - C[t].w = 1.5(xpart(G[0].ne - G[3].sw),0);
+   endfor
+   C[2].c - C[1].c = C[1].c - C[0].c = G[1].c - G[0].c;
+   C[0].c = 0.5[G[0].c,G[3].c];
+ 
+   %% Relate the subfigures.
+   numeric containerConfigurationDistance;
+   containerConfigurationDistance = arrayPartitionDistance;
+ 				% distance between container storage and computer configuration subfigures
+   numeric containerComputationDistance; containerComputationDistance = arrayLayoutDistance;
+ 				% distance between container storage subfigure and computation configuration subfigure
+   numeric arrowComputationDistance; arrowComputationDistance = arrowLayoutDistance;
+ 				% distance between arrow and top of computation configuration, not its label
+ 
+   boxit.figurePlus(btex + etex);
+   boxit.figureArrow(btex $\Big\Downarrow$ etex);
+ 
+   configurationBoundary.w - storageBoundary.e =
+   unit*(containerConfigurationDistance,0); %% HERE
+   figurePlus.c = 0.5[configurationBoundary.w, storageBoundary.e];
+   figureArrow.c = (xpart(0.5[configurationBoundary.e,storageBoundary.w]),
+     ypart(C[1].n) + unit*arrowComputationDistance);
+   
+   0.5[configurationBoundary.se,storageBoundary.sw] - C[1].n =
+   unit*(0,containerComputationDistance);
+   
+   %% Draw the Container Domain Partitioning structures.
+   drawboxed(a);  label.top(btex \begin{tabular}{c} container's\\ domain \end{tabular} etex, a.n);
+   for t = 0 upto 5:
+     drawboxed(p[t]);
+   endfor
+   drawunboxed(pt);
+   label.top(btex partition etex, p[1].n);
+   for t = 0 upto 5:
+     drawboxed(l[t],g[t]);
+   endfor
+   label.top(btex patches etex, g[1].n);
+   z0 = g[2].e + unit*(1,0);
+   drawarrow z0 -- (g[2].e+(iota,0));
+   label.rt(btex \begin{tabular}{l} patch with\\guard cells \end{tabular} etex, z0);
+   drawunboxed(containerPlus,containerArrow);
+   drawboxed(storageBoundary);
+   label.top(btex Partition Container's Domain etex, storageBoundary.n);
+ 
+   %% Draw the Computer Configuration structures.
+   for t = 0 upto 2:
+     drawboxed(c[t]);
+   endfor
+   label.top(btex contexts etex, c[1].n);
+   label.bot(btex \begin{tabular}{c} Each context has memory and\\ processors to execute a program. \end{tabular} etex, c[1].s);
+   drawboxed(configurationBoundary);
+   label.top(btex Computer Configuration etex, configurationBoundary.n);
+ 
+   %% Draw the Computer Computation structures.
+   for t = 0 upto 5:
+     drawboxed(L[t],G[t]);
+   endfor
+   for t = 0 upto 2:
+     drawboxed(C[t]);
+   endfor
+   label.top(btex Layout etex, C[1].n);
+   label.bot(btex Each context can contain several patches. etex, C[1].s);
+ 
+   %% Draw the subfigure relations structures.
+   drawunboxed(figurePlus,figureArrow);
+   label.rt(btex DistributedTag etex, figureArrow.e);
+ endfig;
+ 
+ bye
Index: docs/manual/figures/doof2d.mp
===================================================================
RCS file: doof2d.mp
diff -N doof2d.mp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- doof2d.mp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,257 ----
+ %% Oldham, Jeffrey D.
+ %% 2001Nov26
+ %% Pooma
+ 
+ %% Illustrations for the Tutorial Chapter (Chapter 2)
+ 
+ %% Assumes TEX=latex.
+ 
+ verbatimtex
+ \documentclass[10pt]{article}
+ \begin{document}
+ etex
+ 
+ % Draw a set of grid cells.
+ vardef drawGrid(expr nuCells, unit, llCorner) =
+   for i = 0 upto nuCells-1:
+     for j = 0 upto nuCells-1:
+       draw unitsquare scaled unit shifted (llCorner + unit*(i,j));
+     endfor
+   endfor
+ enddef;
+ 
+ % Label the specified grid, grid cell, or its edge.
+ % Place a value at the center of a grid cell.
+ vardef labelCell(expr lbl, xy, llCorner) =
+   label(lbl, llCorner + unit*(xy + 0.5*(1,1)));
+ enddef;
+ 
+ % Label the bottom of a grid cell.
+ vardef labelCellBottom(expr lbl, xy, llCorner) =
+   label.bot(lbl, llCorner + unit*(xy + 0.5*(1,0)));
+ enddef;
+ 
+ % Label the left side of a grid cell.
+ vardef labelCellLeft(expr lbl, xy, llCorner) =
+   label.lft(lbl, llCorner + unit*(xy + 0.5*(0,1)));
+ enddef;
+ 
+ % Label the top of a grid.
+ vardef labelGrid(expr lbl, nuCells, llCorner) =
+   label.top(lbl, llCorner + unit*(nuCells/2,nuCells));
+ enddef;
+ 
+ %% Global Declarations
+ numeric unit; unit = 0.9cm;	% width or height of an individual grid cell
+ 
+ 
+ %% Initial Configuration.
+ beginfig(201)
+   numeric nuCells; nuCells = 7;	% number of cells in each dimension
+ 				% This number should be odd.
+   % Draw the grid cells.
+   drawGrid(nuCells, unit, origin);
+ 
+   % Label the grid cells' values.
+   for i = 0 upto nuCells-1:
+     for j = 0 upto nuCells-1:
+       if ((i = nuCells div 2) and (j = nuCells div 2)):
+ 	labelCell(btex \footnotesize 1000.0 etex, (i,j), origin);
+       else:
+ 	labelCell(btex \footnotesize 0.0 etex, (i,j), origin);
+       fi
+     endfor
+   endfor
+ 
+   % Label the grid.
+   labelGrid(btex Array \texttt{b}: Initial Configuration etex, nuCells, origin);
+ endfig;
+ 
+ 
+ %% After the first averaging.
+ beginfig(202)
+   numeric nuCells; nuCells = 7;	% number of cells in each dimension
+ 				% This number should be odd.
+   % Draw the grid cells.
+   drawGrid(nuCells, unit, origin);
+ 
+   % Label the grid cells' values.
+   for i = 0, 1, nuCells-2, nuCells-1:
+     for j = 0 upto nuCells-1:
+       labelCell(btex \footnotesize 0.0 etex, (i,j), origin);
+     endfor
+   endfor
+   for j = 0, 1, nuCells-2, nuCells-1:
+     for i = 0 upto nuCells-1:
+       labelCell(btex \footnotesize 0.0 etex, (i,j), origin);
+     endfor
+   endfor
+   for i = (nuCells div 2)-1 upto (nuCells div 2)+1:
+     for j = (nuCells div 2)-1 upto (nuCells div 2)+1:
+       labelCell(btex \footnotesize 111.1 etex, (i, j), origin);
+     endfor
+   endfor
+   
+   % Label the grid.
+   labelGrid(btex Array \texttt{a}: After the first averaging etex, nuCells, origin);
+ endfig;
+ 
+ 
+ %% After the second averaging.
+ beginfig(203)
+   numeric nuCells; nuCells = 7;	% number of cells in each dimension
+ 				% This number should be odd.
+   % Draw the grid cells.
+   drawGrid(nuCells, unit, origin);
+ 
+   % Label the grid cells' values.
+   for i = 0, nuCells-1:
+     for j = 0 upto nuCells-1:
+       labelCell(btex \footnotesize 0.0 etex, (i,j), origin);
+     endfor
+   endfor
+   for j = 0, nuCells-1:
+     for i = 0 upto nuCells-1:
+       labelCell(btex \footnotesize 0.0 etex, (i,j), origin);
+     endfor
+   endfor
+   labelCell(btex \footnotesize 111.1 etex, (3,3), origin);
+   for t = (3,2), (4,3), (3,4), (2,3):
+     labelCell(btex \footnotesize 74.1 etex, t, origin);
+   endfor
+   for t = (2,2), (2,4), (4,4), (4,2):
+     labelCell(btex \footnotesize 49.4 etex, t, origin);
+   endfor
+   for t = (3,1), (5,3), (3,5), (1,3):
+     labelCell(btex \footnotesize 37.0 etex, t, origin);
+   endfor
+   for t = (1,2), (2,1), (4,1), (5,2), (5,4), (4,5), (2,5), (1,4):
+     labelCell(btex \footnotesize 24.7 etex, t, origin);
+   endfor
+   for t = (1,1), (5,1), (5,5), (1,5):
+     labelCell(btex \footnotesize 12.3 etex, t, origin);
+   endfor
+   
+   % Label the grid.
+   labelGrid(btex Array \texttt{b}: After the second averaging etex, nuCells, origin);
+ endfig;
+ 
+ 
+ %% Illustrate addition of arrays.
+ beginfig(210)
+   numeric nuCells; nuCells = 3;	% number of cells in each dimension
+ 				% This number should be odd.
+   numeric operatorWidth; operatorWidth = 1.5;
+   				% horizontal space for an operator as
+   				% a multiple of "unit"
+   
+   %% Determine the locations of the arrays.
+   z0 = origin;
+   z1 = z0 + unit * (nuCells+operatorWidth,0);
+   z2 - z1 = z1 - z0;
+ 
+   %% Draw the grid cells and the operators.
+   for t = 0 upto 2:
+     drawGrid(nuCells, unit, z[t]);
+   endfor
+   label(btex = etex, z1 + unit*(-0.9operatorWidth, 0.5nuCells));
+   label(btex + etex, z2 + unit*(-0.9operatorWidth, 0.5nuCells));
+   
+   %% Label the grid cells' values.
+   % Label b(I,J) grid values.
+   labelCell(btex \normalsize 9 etex, (0,0), z1);
+   labelCell(btex \normalsize 11 etex, (1,0), z1);
+   labelCell(btex \normalsize 13 etex, (2,0), z1);
+   labelCell(btex \normalsize 17 etex, (0,1), z1);
+   labelCell(btex \normalsize 19 etex, (1,1), z1);
+   labelCell(btex \normalsize 21 etex, (2,1), z1);
+   labelCell(btex \normalsize 25 etex, (0,2), z1);
+   labelCell(btex \normalsize 27 etex, (1,2), z1);
+   labelCell(btex \normalsize 29 etex, (2,2), z1);
+   % Label b(I+1,J-1) grid values.
+   labelCell(btex \normalsize 3 etex, (0,0), z2);
+   labelCell(btex \normalsize 5 etex, (1,0), z2);
+   labelCell(btex \normalsize 7 etex, (2,0), z2);
+   labelCell(btex \normalsize 11 etex, (0,1), z2);
+   labelCell(btex \normalsize 13 etex, (1,1), z2);
+   labelCell(btex \normalsize 15 etex, (2,1), z2);
+   labelCell(btex \normalsize 19 etex, (0,2), z2);
+   labelCell(btex \normalsize 21 etex, (1,2), z2);
+   labelCell(btex \normalsize 23 etex, (2,2), z2);
+   % Label b(I,J)+b(I+1,J-1) grid values.
+   labelCell(btex \normalsize 12 etex, (0,0), z0);
+   labelCell(btex \normalsize 16 etex, (1,0), z0);
+   labelCell(btex \normalsize 20 etex, (2,0), z0);
+   labelCell(btex \normalsize 28 etex, (0,1), z0);
+   labelCell(btex \normalsize 32 etex, (1,1), z0);
+   labelCell(btex \normalsize 36 etex, (2,1), z0);
+   labelCell(btex \normalsize 34 etex, (0,2), z0);
+   labelCell(btex \normalsize 38 etex, (1,2), z0);
+   labelCell(btex \normalsize 42 etex, (2,2), z0);
+ 
+   %% Label the indices.
+   % Label b(I,J) grid indices.
+   labelCellBottom(btex \footnotesize 1 etex, (0,0), z1);
+   labelCellBottom(btex \footnotesize 2 etex, (1,0), z1);
+   labelCellBottom(btex \footnotesize 3 etex, (2,0), z1);
+   labelCellLeft(btex \footnotesize 1 etex, (0,0), z1);
+   labelCellLeft(btex \footnotesize 2 etex, (0,1), z1);
+   labelCellLeft(btex \footnotesize 3 etex, (0,2), z1);
+   % Label b(I+1,J-1) grid indices.
+   labelCellBottom(btex \footnotesize 2 etex, (0,0), z2);
+   labelCellBottom(btex \footnotesize 3 etex, (1,0), z2);
+   labelCellBottom(btex \footnotesize 4 etex, (2,0), z2);
+   labelCellLeft(btex \footnotesize 0 etex, (0,0), z2);
+   labelCellLeft(btex \footnotesize 1 etex, (0,1), z2);
+   labelCellLeft(btex \footnotesize 2 etex, (0,2), z2);
+   
+   %% Label the grids.
+   labelGrid(btex $b(I,J)+b(I+1,J-1)$ etex, nuCells, z0);
+   labelGrid(btex $b(I,J)$ etex, nuCells, z1);
+   labelGrid(btex $b(I+1,J-1)$ etex, nuCells, z2);
+ endfig;
+ 
+ 
+ %% Illustrate application of a stencil.
+ beginfig(211)
+   numeric nuCells; nuCells = 5;	% number of cells in each dimension
+   numeric nuStencilCells; nuStencilCells = 3;
+ 				% number of stencil cells in each dimension
+   numeric stencilMultiple; stencilMultiple = 0.1;
+ 				% small multiple to make it visible
+   
+   % Draw the grid cells.
+   drawGrid(nuCells, unit, origin);
+ 
+   % Draw the stencil.
+   draw unitsquare scaled ((nuStencilCells-2stencilMultiple) * unit) shifted (unit*(stencilMultiple*(1,1)+(0,2)));
+   draw (unitsquare scaled ((1-stencilMultiple) * unit) shifted (unit*(0.5*stencilMultiple*(1,1)+(1,3)))) dashed evenly;
+ 
+   % Label the extents.
+   picture lbl;
+   ahlength := 0.4unit;
+   drawarrow unit*(2,4) -- unit*(3,5);
+   lbl = thelabel.lrt(btex \scriptsize upperExtent etex, unit*0.5[(2,4),(3,5)]);
+   unfill bbox lbl; draw lbl;
+   drawarrow unit*(1,3) -- unit*(0,2);
+   lbl := thelabel.lrt(btex \scriptsize lowerExtent etex, unit*0.5[(1,3),(0,2)]);
+   unfill bbox lbl; draw lbl;
+   
+   % Label the indices.
+   labelCellBottom(btex \footnotesize 0 etex, (0,0), origin);
+   labelCellBottom(btex \footnotesize 1 etex, (1,0), origin);
+   labelCellBottom(btex \footnotesize 2 etex, (2,0), origin);
+   labelCellBottom(btex \footnotesize 3 etex, (3,0), origin);
+   labelCellBottom(btex \footnotesize 4 etex, (4,0), origin);
+   labelCellLeft(btex \footnotesize 0 etex, (0,0), origin);
+   labelCellLeft(btex \footnotesize 1 etex, (0,1), origin);
+   labelCellLeft(btex \footnotesize 2 etex, (0,2), origin);
+   labelCellLeft(btex \footnotesize 3 etex, (0,3), origin);
+   labelCellLeft(btex \footnotesize 4 etex, (0,4), origin);
+ 
+   % Label the grid.
+   labelGrid(btex Applying a Stencil to Position (1,3) etex, nuCells, origin);
+   
+ endfig;
+ 
+ bye
Index: docs/manual/figures/makefile
===================================================================
RCS file: makefile
diff -N makefile
*** /dev/null	Fri Mar 23 21:37:44 2001
--- makefile	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,69 ----
+ ### Oldham, Jeffrey D.
+ ### 1997 Dec 26
+ ### misc
+ ###
+ ### LaTeX -> PostScript/PDF/WWW
+ ### XML -> TeX/DVI/PS/PDF
+ 
+ # Definitions for PostScript and WWW Creation
+ TEX=		latex
+ WWWHOMEDIR=	/u/oldham/www
+ LATEX2HTML=	latex2html
+ BASICLATEX2HTMLOPTIONS=	-info "" -no_footnode -no_math -html_version 3.2,math
+ #LATEX2HTMLOPTIONS=	-local_icons -split +1 $(BASICLATEX2HTMLOPTIONS)
+ LATEX2HTMLOPTIONS=	-no_navigation -split 0 $(BASICLATEX2HTMLOPTIONS)
+ MPOST=		mpost
+ 
+ # Definitions for Jade.
+ JADEDIR=		/usr/lib/sgml/stylesheets/docbook
+ PRINTDOCBOOKDSL=	print/docbook.dsl
+ HTMLDOCBOOKDSL=		html/docbook.dsl
+ XML=			dtds/decls/xml.dcl
+ INDEXOPTIONS=		-t 'Index' -i 'index' -g -p
+ 
+ CXXFLAGS= -g -Wall -pedantic -W -Wstrict-prototypes -Wpointer-arith -Wbad-function-cast -Wcast-align -Wconversion -Wnested-externs -Wundef -Winline -static
+ 
+ all: outline.ps
+ 
+ %.all:	%.ps %.pdf %.html
+ 	chmod 644 $*.ps $*.pdf
+ 	mv $*.ps $*.pdf $*
+ 
+ %.dvi:	%.ltx
+ 	$(TEX) $<
+ 	# bibtex $*
+ 	# $(TEX) $<
+ 	$(TEX) $<
+ 
+ %.ps:	%.dvi
+ 	dvips -t letter $< -o
+ 
+ %.pdf.ltx:	%.ltx
+ 	sed -e 's/^%\\usepackage{times}/\\usepackage{times}/' $< > $@
+ 
+ %.pdf:	%.pdf.ps
+ 	ps2pdf $< $@
+ 
+ # This rule assumes index creation.
+ %.dvi:	%.xml genindex.sgm
+ 	jade -D$(JADEDIR) -t sgml -d $(HTMLDOCBOOKDSL) -V html-index $(XML) $<
+ 	perl collateindex.pl $(INDEXOPTIONS) -o genindex.sgm HTML.index
+ 	jade -D$(JADEDIR) -t tex -d $(PRINTDOCBOOKDSL) $(XML) $< && jadetex $*.tex && jadetex $*.tex && jadetex $*.tex
+ 
+ genindex.sgm:
+ 	perl collateindex.pl $(INDEXOPTIONS) -N -o $@
+ 
+ %.html:	%.xml
+ 	jade -D$(JADEDIR) -t sgml -d $(HTMLDOCBOOKDSL) $(XML) $<
+ 
+ %.pdf:	%.xml
+ 	jade -D$(JADEDIR) -t tex -d $(PRINTDOCBOOKDSL) $(XML) $< && pdfjadetex $*.tex && pdfjadetex $*.tex
+ 
+ mproof-%.ps:	%.mp
+ 	declare -x TEX=latex && $(MPOST) $< && tex mproof.tex $*.[0-9]* && dvips mproof.dvi -o $@
+ 
+ %.txt:	%.ltx
+ 	detex $< > $@
+ 
+ clean:
+ 	rm -f *.dvi *.aux *.log *.toc *.bak *.blg *.bbl *.glo *.idx *.lof *.lot *.htm *.mpx mpxerr.tex HTML.index outline.tex
Index: docs/manual/programs/Doof2d-Array-distributed-annotated.patch
===================================================================
RCS file: Doof2d-Array-distributed-annotated.patch
diff -N Doof2d-Array-distributed-annotated.patch
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-distributed-annotated.patch	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,162 ----
+ *** Doof2d-Array-distributed.cpp	Wed Nov 28 07:46:56 2001
+ --- Doof2d-Array-distributed-annotated.cpp	Wed Nov 28 07:53:31 2001
+ ***************
+ *** 1,4 ****
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+ --- 1,5 ----
+ ! <programlisting id="tutorial-array_distributed-doof2d-program" linenumbering="numbered" format="linespecific">
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+ ***************
+ *** 15,19 ****
+     // (i,j).  The "C" template parameter permits use of this stencil
+     // operator with both Arrays and Fields.
+ !   template <class C>
+     inline
+     typename C::Element_t
+ --- 16,20 ----
+     // (i,j).  The "C" template parameter permits use of this stencil
+     // operator with both Arrays and Fields.
+ !   template <class C>
+     inline
+     typename C::Element_t
+ ***************
+ *** 40,52 ****
+     Pooma::initialize(argc,argv);
+     
+ !   // Ask the user for the number of processors.
+     long nuProcessors;
+ !   std::cout << "Please enter the number of processors: ";
+ !   std::cin >> nuProcessors;
+   
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ --- 41,53 ----
+     Pooma::initialize(argc,argv);
+     
+ !   // Ask the user for the number of processors.  <co id="tutorial-array_distributed-doof2d-nuprocessors"></co>
+     long nuProcessors;
+ !   std::cout << "Please enter the number of processors: ";
+ !   std::cin >> nuProcessors;
+   
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ ***************
+ *** 54,67 ****
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+     // Set up interior domains [1,n-1) x [1,n-1) for computation.
+ !   Interval<1> I(1,n-2);
+ !   Interval<2> interiorDomain(I,I);
+   
+     // Create the distributed arrays.
+ --- 55,68 ----
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+     // Set up interior domains [1,n-1) x [1,n-1) for computation.
+ !   Interval<1> I(1,n-2);
+ !   Interval<2> interiorDomain(I,I);
+   
+     // Create the distributed arrays.
+ ***************
+ *** 70,85 ****
+     // dimension.  Guard layers optimize communication between patches.
+     // Internal guards surround each patch.  External guards surround
+ !   // the entire array domain.
+ !   UniformGridPartition<2> partition(Loc<2>(nuProcessors, nuProcessors),
+ ! 				    GuardLayers<2>(1),  // internal
+ ! 				    GuardLayers<2>(0)); // external
+ !   UniformGridLayout<2> layout(vertDomain, partition, DistributedTag());
+   
+     // The template parameters indicate 2 dimensions and a 'double'
+     // element type.  MultiPatch indicates multiple computation patches,
+     // i.e., distributed computation.  The UniformTag indicates the
+ !   // patches should have the same size.  Each patch has Brick type.
+ !   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > a(layout);
+ !   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > b(layout);
+   
+     // Set up the initial conditions.
+ --- 71,86 ----
+     // dimension.  Guard layers optimize communication between patches.
+     // Internal guards surround each patch.  External guards surround
+ !   // the entire array domain.  <co id="tutorial-array_distributed-doof2d-layout"></co>
+ !   UniformGridPartition<2> partition(Loc<2>(nuProcessors, nuProcessors),
+ ! 				    GuardLayers<2>(1),  // internal
+ ! 				    GuardLayers<2>(0)); // external
+ !   UniformGridLayout<2> layout(vertDomain, partition, DistributedTag());
+   
+     // The template parameters indicate 2 dimensions and a 'double'
+     // element type.  MultiPatch indicates multiple computation patches,
+     // i.e., distributed computation.  The UniformTag indicates the
+ !   // patches should have the same size.  Each patch has Brick type.  <co id="tutorial-array_distributed-doof2d-remote"></co>
+ !   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > a(layout);
+ !   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > b(layout);
+   
+     // Set up the initial conditions.
+ ***************
+ *** 89,97 ****
+   
+     // Create the stencil performing the computation.
+ !   Stencil<DoofNinePt> stencil;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.
+       a(interiorDomain) = stencil(b, interiorDomain);
+   
+ --- 90,98 ----
+   
+     // Create the stencil performing the computation.
+ !   Stencil<DoofNinePt> stencil;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.  <co id="tutorial-array_distributed-doof2d-first_write"></co>
+       a(interiorDomain) = stencil(b, interiorDomain);
+   
+ ***************
+ *** 102,106 ****
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ --- 103,107 ----
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ ***************
+ *** 110,111 ****
+ --- 111,113 ----
+     return EXIT_SUCCESS;
+   }
+ + </programlisting>
Index: docs/manual/programs/Doof2d-Array-element-annotated.patch
===================================================================
RCS file: Doof2d-Array-element-annotated.patch
diff -N Doof2d-Array-element-annotated.patch
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-element-annotated.patch	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,144 ----
+ *** Doof2d-Array-element.cpp	Tue Nov 27 11:04:04 2001
+ --- Doof2d-Array-element-annotated.cpp	Tue Nov 27 12:06:32 2001
+ ***************
+ *** 1,5 ****
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+ ! #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+   // Doof2d: Pooma Arrays, element-wise implementation
+ --- 1,6 ----
+ ! <programlisting id="tutorial-array_elementwise-doof2d-program" linenumbering="numbered" format="linespecific">
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+ ! #include "Pooma/Arrays.h"	// has Pooma's Array  <co id="tutorial-array_elementwise-doof2d-header"></co>
+   
+   // Doof2d: Pooma Arrays, element-wise implementation
+ ***************
+ *** 7,17 ****
+   int main(int argc, char *argv[])
+   {
+ !   // Prepare the Pooma library for execution.
+     Pooma::initialize(argc,argv);
+     
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ --- 8,18 ----
+   int main(int argc, char *argv[])
+   {
+ !   // Prepare the Pooma library for execution.  <co id="tutorial-array_elementwise-doof2d-pooma_initialize"></co>
+     Pooma::initialize(argc,argv);
+     
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ ***************
+ *** 19,37 ****
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+ !   // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+ !   // Create the arrays.
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ !   // All grid values should be zero except for the central value.
+     a = b = 0.0;
+     b(n/2,n/2) = 1000.0;
+ --- 20,38 ----
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+ !   // Specify the arrays' domains [0,n) x [0,n).  <co id="tutorial-array_elementwise-doof2d-domain"></co>
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+ !   // Create the arrays.  <co id="tutorial-array_elementwise-doof2d-array_creation"></co>
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ !   // All grid values should be zero except for the central value.  <co id="tutorial-array_elementwise-doof2d-initialization"></co>
+     a = b = 0.0;
+     b(n/2,n/2) = 1000.0;
+ ***************
+ *** 41,49 ****
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+       // Read from b.  Write to a.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+ !         a(i,j) = weight *
+             (b(i+1,j+1) + b(i+1,j  ) + b(i+1,j-1) +
+              b(i  ,j+1) + b(i  ,j  ) + b(i  ,j-1) +
+ --- 42,50 ----
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+       // Read from b.  Write to a.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+ !         a(i,j) = weight *  <co id="tutorial-array_elementwise-doof2d-first_write"></co>
+             (b(i+1,j+1) + b(i+1,j  ) + b(i+1,j-1) +
+              b(i  ,j+1) + b(i  ,j  ) + b(i  ,j-1) +
+ ***************
+ *** 51,56 ****
+   
+       // Read from a.  Write to b.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           b(i,j) = weight *
+             (a(i+1,j+1) + a(i+1,j  ) + a(i+1,j-1) +
+ --- 52,57 ----
+   
+       // Read from a.  Write to b.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           b(i,j) = weight *
+             (a(i+1,j+1) + a(i+1,j  ) + a(i+1,j-1) +
+ ***************
+ *** 60,70 ****
+   
+     // Print out the final central value.
+ !   Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+ !   // The arrays are automatically deallocated.
+   
+ !   // Tell the Pooma library execution has finished.
+     Pooma::finalize();
+     return EXIT_SUCCESS;
+   }
+ --- 61,72 ----
+   
+     // Print out the final central value.
+ !   Pooma::blockAndEvaluate();	// Ensure all computation has finished.  <co id="tutorial-array_elementwise-doof2d-blockAndEvaluate"></co>
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+ !   // The arrays are automatically deallocated.  <co id="tutorial-array_elementwise-doof2d-deallocation"></co>
+   
+ !   // Tell the Pooma library execution has finished.  <co id="tutorial-array_elementwise-doof2d-pooma_finish"></co>
+     Pooma::finalize();
+     return EXIT_SUCCESS;
+   }
+ + </programlisting>
Index: docs/manual/programs/Doof2d-Array-parallel-annotated.patch
===================================================================
RCS file: Doof2d-Array-parallel-annotated.patch
diff -N Doof2d-Array-parallel-annotated.patch
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-parallel-annotated.patch	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,106 ----
+ *** Doof2d-Array-parallel.cpp	Tue Nov 27 13:00:09 2001
+ --- Doof2d-Array-parallel-annotated.cpp	Tue Nov 27 14:07:07 2001
+ ***************
+ *** 1,4 ****
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+ --- 1,5 ----
+ ! <programlisting id="tutorial-array_parallel-doof2d-program" linenumbering="numbered" format="linespecific">
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+ ***************
+ *** 12,17 ****
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ --- 13,18 ----
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ ***************
+ *** 19,38 ****
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+ !   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+ !   Interval<1> I(1,n-2);
+ !   Interval<1> J(1,n-2);
+   
+     // Create the arrays.
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ --- 20,39 ----
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+ !   // Set up interior domains [1,n-1) x [1,n-1) for computation.  <co id="tutorial-array_parallel-doof2d-innerdomain"></co>
+ !   Interval<1> I(1,n-2);
+ !   Interval<1> J(1,n-2);
+   
+     // Create the arrays.
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ ***************
+ *** 45,50 ****
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.
+       a(I,J) = weight *
+         (b(I+1,J+1) + b(I+1,J  ) + b(I+1,J-1) +
+ --- 46,51 ----
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.  <co id="tutorial-array_parallel-doof2d-first_write"></co>
+       a(I,J) = weight *
+         (b(I+1,J+1) + b(I+1,J  ) + b(I+1,J-1) +
+ ***************
+ *** 61,65 ****
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ --- 62,66 ----
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ ***************
+ *** 69,70 ****
+ --- 70,72 ----
+     return EXIT_SUCCESS;
+   }
+ + </programlisting>
Index: docs/manual/programs/Doof2d-Array-stencil-annotated.patch
===================================================================
RCS file: Doof2d-Array-stencil-annotated.patch
diff -N Doof2d-Array-stencil-annotated.patch
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-stencil-annotated.patch	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,152 ----
+ *** Doof2d-Array-stencil.cpp	Tue Nov 27 17:23:41 2001
+ --- Doof2d-Array-stencil-annotated.cpp	Tue Nov 27 17:36:13 2001
+ ***************
+ *** 1,9 ****
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+   // Doof2d: Pooma Arrays, stencil implementation
+   
+ ! // Define the stencil class performing the computation.
+   class DoofNinePt
+   {
+ --- 1,10 ----
+ ! <programlisting id="tutorial-array_stencil-doof2d-program" linenumbering="numbered" format="linespecific">
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   #include "Pooma/Arrays.h"	// has Pooma's Array
+   
+   // Doof2d: Pooma Arrays, stencil implementation
+   
+ ! // Define the stencil class performing the computation.  <co id="tutorial-array_stencil-doof2d-stencil"></co>
+   class DoofNinePt
+   {
+ ***************
+ *** 14,19 ****
+     // This stencil operator is applied to each interior domain position
+     // (i,j).  The "C" template parameter permits use of this stencil
+ !   // operator with both Arrays and Fields.
+ !   template <class C>
+     inline
+     typename C::Element_t
+ --- 15,20 ----
+     // This stencil operator is applied to each interior domain position
+     // (i,j).  The "C" template parameter permits use of this stencil
+ !   // operator with both Arrays and Fields.  <co id="tutorial-array_stencil-doof2d-stencil_operator"></co>
+ !   template <class C>
+     inline
+     typename C::Element_t
+ ***************
+ *** 26,30 ****
+     }
+   
+ !   inline int lowerExtent(int) const { return 1; }
+     inline int upperExtent(int) const { return 1; }
+   
+ --- 27,31 ----
+     }
+   
+ !   inline int lowerExtent(int) const { return 1; }  <co id="tutorial-array_stencil-doof2d-stencil_extent"></co>
+     inline int upperExtent(int) const { return 1; }
+   
+ ***************
+ *** 42,47 ****
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ --- 43,48 ----
+     // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ ***************
+ *** 49,68 ****
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+     // Set up interior domains [1,n-1) x [1,n-1) for computation.
+ !   Interval<1> I(1,n-2);
+ !   Interval<2> interiorDomain(I,I);
+   
+     // Create the arrays.
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ --- 50,69 ----
+     // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+     // Specify the arrays' domains [0,n) x [0,n).
+ !   Interval<1> N(0, n-1);
+ !   Interval<2> vertDomain(N, N);
+   
+     // Set up interior domains [1,n-1) x [1,n-1) for computation.
+ !   Interval<1> I(1,n-2);
+ !   Interval<2> interiorDomain(I,I);
+   
+     // Create the arrays.
+     // The template parameters indicate 2 dimensions, a 'double' element
+     // type, and ordinary 'Brick' storage.
+ !   Array<2, double, Brick> a(vertDomain);
+ !   Array<2, double, Brick> b(vertDomain);
+   
+     // Set up the initial conditions.
+ ***************
+ *** 71,80 ****
+     b(n/2,n/2) = 1000.0;
+   
+ !   // Create the stencil performing the computation.
+ !   Stencil<DoofNinePt> stencil;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.
+       a(interiorDomain) = stencil(b, interiorDomain);
+   
+ --- 72,81 ----
+     b(n/2,n/2) = 1000.0;
+   
+ !   // Create the stencil performing the computation.  <co id="tutorial-array_stencil-doof2d-stencil_creation"></co>
+ !   Stencil<DoofNinePt> stencil;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.  <co id="tutorial-array_stencil-doof2d-first_write"></co>
+       a(interiorDomain) = stencil(b, interiorDomain);
+   
+ ***************
+ *** 85,89 ****
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ --- 86,90 ----
+     // Print out the final central value.
+     Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+ !   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+   
+     // The arrays are automatically deallocated.
+ ***************
+ *** 93,94 ****
+ --- 94,96 ----
+     return EXIT_SUCCESS;
+   }
+ + </programlisting>
Index: docs/manual/programs/Doof2d-C-element-annotated.patch
===================================================================
RCS file: Doof2d-C-element-annotated.patch
diff -N Doof2d-C-element-annotated.patch
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-C-element-annotated.patch	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,150 ----
+ *** Doof2d-C-element.cpp	Tue Nov 27 08:36:38 2001
+ --- Doof2d-C-element-annotated.cpp	Tue Nov 27 12:08:03 2001
+ ***************
+ *** 1,4 ****
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   
+   // Doof2d: C-like, element-wise implementation
+ --- 1,5 ----
+ ! <programlisting id="tutorial-hand_coded-doof2d-program" linenumbering="numbered" format="linespecific">
+ ! #include <iostream>		// has std::cout, ...
+ ! #include <stdlib.h>		// has EXIT_SUCCESS
+   
+   // Doof2d: C-like, element-wise implementation
+ ***************
+ *** 6,30 ****
+   int main()
+   {
+ !   // Ask the user for the number of averagings.
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ !   // Use two-dimensional grids of values.
+     double **a;
+     double **b;
+   
+     // Ask the user for the number n of elements along one dimension of
+ !   // the grid.
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+ !   // Allocate the arrays.
+     typedef double* doublePtr;
+     a = new doublePtr[n];
+     b = new doublePtr[n];
+ !   for (int i = 0; i < n; i++) {
+       a[i] = new double[n];
+       b[i] = new double[n];
+ --- 7,31 ----
+   int main()
+   {
+ !   // Ask the user for the number of averagings.  <co id="tutorial-hand_coded-doof2d-nuaveragings"></co>
+     long nuAveragings, nuIterations;
+ !   std::cout << "Please enter the number of averagings: ";
+ !   std::cin >> nuAveragings;
+     nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+   
+ !   // Use two-dimensional grids of values.  <co id="tutorial-hand_coded-doof2d-array_storage"></co>
+     double **a;
+     double **b;
+   
+     // Ask the user for the number n of elements along one dimension of
+ !   // the grid.  <co id="tutorial-hand_coded-doof2d-grid_size"></co>
+     long n;
+ !   std::cout << "Please enter the array size: ";
+ !   std::cin >> n;
+   
+ !   // Allocate the arrays.  <co id="tutorial-hand_coded-doof2d-allocation"></co>
+     typedef double* doublePtr;
+     a = new doublePtr[n];
+     b = new doublePtr[n];
+ !   for (int i = 0; i < n; i++) {
+       a[i] = new double[n];
+       b[i] = new double[n];
+ ***************
+ *** 32,49 ****
+   
+     // Set up the initial conditions.
+ !   // All grid values should be zero except for the central value.
+ !   for (int j = 0; j < n; j++)
+ !     for (int i = 0; i < n; i++)
+         a[i][j] = b[i][j] = 0.0;
+     b[n/2][n/2] = 1000.0;
+   
+ !   // In the average, weight element with this value.
+     const double weight = 1.0/9.0;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           a[i][j] = weight *
+             (b[i+1][j+1] + b[i+1][j  ] + b[i+1][j-1] +
+ --- 33,50 ----
+   
+     // Set up the initial conditions.
+ !   // All grid values should be zero except for the central value.  <co id="tutorial-hand_coded-doof2d-initialization"></co>
+ !   for (int j = 0; j < n; j++)
+ !     for (int i = 0; i < n; i++)
+         a[i][j] = b[i][j] = 0.0;
+     b[n/2][n/2] = 1000.0;
+   
+ !   // In the average, weight element with this value.  <co id="tutorial-hand_coded-doof2d-constants"></co>
+     const double weight = 1.0/9.0;
+   
+     // Perform the simulation.
+ !   for (int k = 0; k < nuIterations; ++k) {
+ !     // Read from b.  Write to a.  <co id="tutorial-hand_coded-doof2d-first_write"></co>
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           a[i][j] = weight *
+             (b[i+1][j+1] + b[i+1][j  ] + b[i+1][j-1] +
+ ***************
+ *** 51,57 ****
+              b[i-1][j+1] + b[i-1][j  ] + b[i-1][j-1]);
+   
+ !     // Read from a.  Write to b.
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           b[i][j] = weight *
+             (a[i+1][j+1] + a[i+1][j  ] + a[i+1][j-1] +
+ --- 52,58 ----
+              b[i-1][j+1] + b[i-1][j  ] + b[i-1][j-1]);
+   
+ !     // Read from a.  Write to b.  <co id="tutorial-hand_coded-doof2d-second_write"></co>
+ !     for (int j = 1; j < n-1; j++)
+ !       for (int i = 1; i < n-1; i++)
+           b[i][j] = weight *
+             (a[i+1][j+1] + a[i+1][j  ] + a[i+1][j-1] +
+ ***************
+ *** 60,68 ****
+     }
+   
+ !   // Print out the final central value.
+ !   std::cout << (nuAveragings % 2 ? a[n/2][n/2] : b[n/2][n/2]) << std::endl;
+   
+ !   // Deallocate the arrays.
+ !   for (int i = 0; i < n; i++) {
+       delete [] a[i];
+       delete [] b[i];
+ --- 61,69 ----
+     }
+   
+ !   // Print out the final central value.  <co id="tutorial-hand_coded-doof2d-answer"></co>
+ !   std::cout << (nuAveragings % 2 ? a[n/2][n/2] : b[n/2][n/2]) << std::endl;
+   
+ !   // Deallocate the arrays.  <co id="tutorial-hand_coded-doof2d-deallocation"></co>
+ !   for (int i = 0; i < n; i++) {
+       delete [] a[i];
+       delete [] b[i];
+ ***************
+ *** 73,74 ****
+ --- 74,76 ----
+     return EXIT_SUCCESS;
+   }
+ + </programlisting>
Index: docs/manual/programs/makefile
===================================================================
RCS file: makefile
diff -N makefile
*** /dev/null	Fri Mar 23 21:37:44 2001
--- makefile	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,12 ----
+ ### Oldham, Jeffrey D.
+ ### 2001Nov27
+ ### Pooma
+ ###
+ ### Produce Annotated Source Code
+ 
+ all: Doof2d-C-element-annotated.cpp Doof2d-Array-element-annotated.cpp \
+      Doof2d-Array-parallel-annotated.cpp Doof2d-Array-stencil-annotated.cpp \
+      Doof2d-Array-distributed-annotated.cpp
+ 
+ %-annotated.cpp: %-annotated.patch %.cpp
+ 	patch -o $@ < $<
Index: examples/Manual/Doof2d/Doof2d-Array-distributed.cpp
===================================================================
RCS file: Doof2d-Array-distributed.cpp
diff -N Doof2d-Array-distributed.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-distributed.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,111 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Arrays.h"	// has Pooma's Array
+ 
+ // Doof2d: Pooma Arrays, stencil, multiple processor implementation
+ 
+ // Define the stencil class performing the computation.
+ class DoofNinePt
+ {
+ public:
+   // Initialize the constant average weighting.
+   DoofNinePt() : weight(1.0/9.0) {}
+ 
+   // This stencil operator is applied to each interior domain position
+   // (i,j).  The "C" template parameter permits use of this stencil
+   // operator with both Arrays and Fields.
+   template <class C>
+   inline
+   typename C::Element_t
+   operator()(const C& x, int i, int j) const
+   {
+     return ( weight *
+              ( x.read(i+1,j+1) + x.read(i+1,j  ) + x.read(i+1,j-1) +
+                x.read(i  ,j+1) + x.read(i  ,j  ) + x.read(i  ,j-1) +
+                x.read(i-1,j+1) + x.read(i-1,j  ) + x.read(i-1,j-1) ) );
+   }
+ 
+   inline int lowerExtent(int) const { return 1; }
+   inline int upperExtent(int) const { return 1; }
+ 
+ private:
+ 
+   // In the average, weight element with this value.
+   const double weight;
+ };
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // Ask the user for the number of processors.
+   long nuProcessors;
+   std::cout << "Please enter the number of processors: ";
+   std::cin >> nuProcessors;
+ 
+   // Ask the user for the number of averagings.
+   long nuAveragings, nuIterations;
+   std::cout << "Please enter the number of averagings: ";
+   std::cin >> nuAveragings;
+   nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the arrays' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+   Interval<1> I(1,n-2);
+   Interval<2> interiorDomain(I,I);
+ 
+   // Create the distributed arrays.
+   // Partition the arrays' domains uniformly, i.e., each patch has the
+   // same size.  The first parameter tells how many patches for each
+   // dimension.  Guard layers optimize communication between patches.
+   // Internal guards surround each patch.  External guards surround
+   // the entire array domain.
+   UniformGridPartition<2> partition(Loc<2>(nuProcessors, nuProcessors),
+ 				    GuardLayers<2>(1),  // internal
+ 				    GuardLayers<2>(0)); // external
+   UniformGridLayout<2> layout(vertDomain, partition, DistributedTag());
+ 
+   // The template parameters indicate 2 dimensions and a 'double'
+   // element type.  MultiPatch indicates multiple computation patches,
+   // i.e., distributed computation.  The UniformTag indicates the
+   // patches should have the same size.  Each patch has Brick type.
+   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > a(layout);
+   Array<2, double, MultiPatch<UniformTag, Remote<Brick> > > b(layout);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // Create the stencil performing the computation.
+   Stencil<DoofNinePt> stencil;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     a(interiorDomain) = stencil(b, interiorDomain);
+ 
+     // Read from a.  Write to b.
+     b(interiorDomain) = stencil(a, interiorDomain);
+   }
+ 
+   // Print out the final central value.
+   Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+ 
+   // The arrays are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-Array-element.cpp
===================================================================
RCS file: Doof2d-Array-element.cpp
diff -N Doof2d-Array-element.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-element.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,70 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Arrays.h"	// has Pooma's Array
+ 
+ // Doof2d: Pooma Arrays, element-wise implementation
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // Ask the user for the number of averagings.
+   long nuAveragings, nuIterations;
+   std::cout << "Please enter the number of averagings: ";
+   std::cin >> nuAveragings;
+   nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the arrays' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Create the arrays.
+   // The template parameters indicate 2 dimensions, a 'double' element
+   // type, and ordinary 'Brick' storage.
+   Array<2, double, Brick> a(vertDomain);
+   Array<2, double, Brick> b(vertDomain);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // In the average, weight element with this value.
+   const double weight = 1.0/9.0;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     for (int j = 1; j < n-1; j++)
+       for (int i = 1; i < n-1; i++)
+         a(i,j) = weight *
+           (b(i+1,j+1) + b(i+1,j  ) + b(i+1,j-1) +
+            b(i  ,j+1) + b(i  ,j  ) + b(i  ,j-1) +
+            b(i-1,j+1) + b(i-1,j  ) + b(i-1,j-1));
+ 
+     // Read from a.  Write to b.
+     for (int j = 1; j < n-1; j++)
+       for (int i = 1; i < n-1; i++)
+         b(i,j) = weight *
+           (a(i+1,j+1) + a(i+1,j  ) + a(i+1,j-1) +
+            a(i  ,j+1) + a(i  ,j  ) + a(i  ,j-1) +
+            a(i-1,j+1) + a(i-1,j  ) + a(i-1,j-1));
+   }
+ 
+   // Print out the final central value.
+   Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+ 
+   // The arrays are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-Array-parallel.cpp
===================================================================
RCS file: Doof2d-Array-parallel.cpp
diff -N Doof2d-Array-parallel.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-parallel.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,70 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Arrays.h"	// has Pooma's Array
+ 
+ // Doof2d: Pooma Arrays, data-parallel implementation
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // Ask the user for the number of averagings.
+   long nuAveragings, nuIterations;
+   std::cout << "Please enter the number of averagings: ";
+   std::cin >> nuAveragings;
+   nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the arrays' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+   Interval<1> I(1,n-2);
+   Interval<1> J(1,n-2);
+ 
+   // Create the arrays.
+   // The template parameters indicate 2 dimensions, a 'double' element
+   // type, and ordinary 'Brick' storage.
+   Array<2, double, Brick> a(vertDomain);
+   Array<2, double, Brick> b(vertDomain);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // In the average, weight element with this value.
+   const double weight = 1.0/9.0;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     a(I,J) = weight *
+       (b(I+1,J+1) + b(I+1,J  ) + b(I+1,J-1) +
+        b(I  ,J+1) + b(I  ,J  ) + b(I  ,J-1) +
+        b(I-1,J+1) + b(I-1,J  ) + b(I-1,J-1));
+ 
+     // Read from a.  Write to b.
+     b(I,J) = weight *
+       (a(I+1,J+1) + a(I+1,J  ) + a(I+1,J-1) +
+        a(I  ,J+1) + a(I  ,J  ) + a(I  ,J-1) +
+        a(I-1,J+1) + a(I-1,J  ) + a(I-1,J-1));
+   }
+ 
+   // Print out the final central value.
+   Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+ 
+   // The arrays are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-Array-stencil.cpp
===================================================================
RCS file: Doof2d-Array-stencil.cpp
diff -N Doof2d-Array-stencil.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Array-stencil.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,94 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Arrays.h"	// has Pooma's Array
+ 
+ // Doof2d: Pooma Arrays, stencil implementation
+ 
+ // Define the stencil class performing the computation.
+ class DoofNinePt
+ {
+ public:
+   // Initialize the constant average weighting.
+   DoofNinePt() : weight(1.0/9.0) {}
+ 
+   // This stencil operator is applied to each interior domain position
+   // (i,j).  The "C" template parameter permits use of this stencil
+   // operator with both Arrays and Fields.
+   template <class C>
+   inline
+   typename C::Element_t
+   operator()(const C& c, int i, int j) const
+   {
+     return ( weight *
+              ( c.read(i+1,j+1) + c.read(i+1,j  ) + c.read(i+1,j-1) +
+                c.read(i  ,j+1) + c.read(i  ,j  ) + c.read(i  ,j-1) +
+                c.read(i-1,j+1) + c.read(i-1,j  ) + c.read(i-1,j-1) ) );
+   }
+ 
+   inline int lowerExtent(int) const { return 1; }
+   inline int upperExtent(int) const { return 1; }
+ 
+ private:
+ 
+   // In the average, weight element with this value.
+   const double weight;
+ };
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // Ask the user for the number of averagings.
+   long nuAveragings, nuIterations;
+   std::cout << "Please enter the number of averagings: ";
+   std::cin >> nuAveragings;
+   nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the arrays' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+   Interval<1> I(1,n-2);
+   Interval<2> interiorDomain(I,I);
+ 
+   // Create the arrays.
+   // The template parameters indicate 2 dimensions, a 'double' element
+   // type, and ordinary 'Brick' storage.
+   Array<2, double, Brick> a(vertDomain);
+   Array<2, double, Brick> b(vertDomain);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // Create the stencil performing the computation.
+   Stencil<DoofNinePt> stencil;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     a(interiorDomain) = stencil(b, interiorDomain);
+ 
+     // Read from a.  Write to b.
+     b(interiorDomain) = stencil(a, interiorDomain);
+   }
+ 
+   // Print out the final central value.
+   Pooma::blockAndEvaluate();	// Ensure all computation has finished.
+   std::cout << (nuAveragings % 2 ? a(n/2,n/2) : b(n/2,n/2)) << std::endl;
+ 
+   // The arrays are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-C-element.cpp
===================================================================
RCS file: Doof2d-C-element.cpp
diff -N Doof2d-C-element.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-C-element.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,74 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ 
+ // Doof2d: C-like, element-wise implementation
+ 
+ int main()
+ {
+   // Ask the user for the number of averagings.
+   long nuAveragings, nuIterations;
+   std::cout << "Please enter the number of averagings: ";
+   std::cin >> nuAveragings;
+   nuIterations = (nuAveragings+1)/2; // Each iteration performs two averagings.
+ 
+   // Use two-dimensional grids of values.
+   double **a;
+   double **b;
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Allocate the arrays.
+   typedef double* doublePtr;
+   a = new doublePtr[n];
+   b = new doublePtr[n];
+   for (int i = 0; i < n; i++) {
+     a[i] = new double[n];
+     b[i] = new double[n];
+   }
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   for (int j = 0; j < n; j++)
+     for (int i = 0; i < n; i++)
+       a[i][j] = b[i][j] = 0.0;
+   b[n/2][n/2] = 1000.0;
+ 
+   // In the average, weight element with this value.
+   const double weight = 1.0/9.0;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     for (int j = 1; j < n-1; j++)
+       for (int i = 1; i < n-1; i++)
+         a[i][j] = weight *
+           (b[i+1][j+1] + b[i+1][j  ] + b[i+1][j-1] +
+            b[i  ][j+1] + b[i  ][j  ] + b[i  ][j-1] +
+            b[i-1][j+1] + b[i-1][j  ] + b[i-1][j-1]);
+ 
+     // Read from a.  Write to b.
+     for (int j = 1; j < n-1; j++)
+       for (int i = 1; i < n-1; i++)
+         b[i][j] = weight *
+           (a[i+1][j+1] + a[i+1][j  ] + a[i+1][j-1] +
+            a[i  ][j+1] + a[i  ][j  ] + a[i  ][j-1] +
+            a[i-1][j+1] + a[i-1][j  ] + a[i-1][j-1]);
+   }
+ 
+   // Print out the final central value.
+   std::cout << (nuAveragings % 2 ? a[n/2][n/2] : b[n/2][n/2]) << std::endl;
+ 
+   // Deallocate the arrays.
+   for (int i = 0; i < n; i++) {
+     delete [] a[i];
+     delete [] b[i];
+   }
+   delete [] a;
+   delete [] b;
+ 
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-Field-distributed.cpp
===================================================================
RCS file: Doof2d-Field-distributed.cpp
diff -N Doof2d-Field-distributed.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Field-distributed.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,87 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Fields.h"	// has Pooma's Field
+ 
+ // Doof2d: Pooma Fields, data-parallel, multiple processor implementation
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // nuIterations is the number of simulation iterations.
+   const int nuIterations = 10/2;
+ 
+   // In the average, weight element with this value.
+   const double weight = 1.0/9.0;
+ 
+   // nuProcessors is the number of processors along one dimension.
+   const int nuProcessors = 2;
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the fields' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+   Interval<1> I(1,n-2);
+   Interval<1> J(1,n-2);
+ 
+   // Partition the fields' domains uniformly, i.e., each patch has the
+   // same size.  The first parameter tells how many patches for each
+   // dimension.  Guard layers optimize communication between patches.
+   // Internal guards surround each patch.  External guards surround
+   // the entire field domain.
+   UniformGridPartition<2> partition(Loc<2>(nuProcessors, nuProcessors),
+ 				    GuardLayers<2>(1),  // internal
+ 				    GuardLayers<2>(0)); // external
+   UniformGridLayout<2> layout(vertDomain, partition, DistributedTag());
+ 
+   // Specify the fields' mesh, i.e., its spatial extent, and its
+   // centering type.
+   UniformRectilinearMesh<2> mesh(layout, Vector<2>(0.0), Vector<2>(1.0, 1.0));
+   Centering<2> cell = canonicalCentering<2>(CellType, Continuous, AllDim);
+ 
+   // The template parameters indicate a mesh and a 'double'
+   // element type.  MultiPatch indicates multiple computation patches,
+   // i.e., distributed computation.  The UniformTag indicates the
+   // patches should have the same size.  Each patch has Brick type.
+   Field<UniformRectilinearMesh<2>, double, MultiPatch<UniformTag,
+     Remote<Brick> > > a(cell, layout, mesh);
+   Field<UniformRectilinearMesh<2>, double, MultiPatch<UniformTag,
+     Remote<Brick> > > b(cell, layout, mesh);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     a(I,J) = weight *
+       (b(I+1,J+1) + b(I+1,J  ) + b(I+1,J-1) +
+        b(I  ,J+1) + b(I  ,J  ) + b(I  ,J-1) +
+        b(I-1,J+1) + b(I-1,J  ) + b(I-1,J-1));
+ 
+     // Read from a.  Write to b.
+     b(I,J) = weight *
+       (a(I+1,J+1) + a(I+1,J  ) + a(I+1,J-1) +
+        a(I  ,J+1) + a(I  ,J  ) + a(I  ,J-1) +
+        a(I-1,J+1) + a(I-1,J  ) + a(I-1,J-1));
+   }
+ 
+   // Print out the final central value.
+   std::cout << b(n/2,n/2) << std::endl;
+ 
+   // The fields are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/Doof2d-Field-parallel.cpp
===================================================================
RCS file: Doof2d-Field-parallel.cpp
diff -N Doof2d-Field-parallel.cpp
*** /dev/null	Fri Mar 23 21:37:44 2001
--- Doof2d-Field-parallel.cpp	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,72 ----
+ #include <iostream>		// has std::cout, ...
+ #include <stdlib.h>		// has EXIT_SUCCESS
+ #include "Pooma/Fields.h"	// has Pooma's Field
+ 
+ // Doof2d: Pooma Fields, data-parallel implementation
+ 
+ int main(int argc, char *argv[])
+ {
+   // Prepare the Pooma library for execution.
+   Pooma::initialize(argc,argv);
+   
+   // nuIterations is the number of simulation iterations.
+   const int nuIterations = 10/2;
+ 
+   // In the average, weight element with this value.
+   const double weight = 1.0/9.0;
+ 
+   // Ask the user for the number n of elements along one dimension of
+   // the grid.
+   long n;
+   std::cout << "Please enter the array size: ";
+   std::cin >> n;
+ 
+   // Specify the fields' domains [0,n) x [0,n).
+   Interval<1> N(0, n-1);
+   Interval<2> vertDomain(N, N);
+ 
+   // Set up interior domains [1,n-1) x [1,n-1) for computation.
+   Interval<1> I(1,n-2);
+   Interval<1> J(1,n-2);
+ 
+   // Specify the fields' mesh, i.e., its spatial extent, and its
+   // centering type.
+   DomainLayout<2> layout(vertDomain);
+   UniformRectilinearMesh<2> mesh(layout, Vector<2>(0.0), Vector<2>(1.0, 1.0));
+   Centering<2> cell = canonicalCentering<2>(CellType, Continuous, AllDim);
+ 
+   // Create the fields.
+   // The template parameters indicate a mesh, a 'double' element
+   // type, and ordinary 'Brick' storage.
+   Field<UniformRectilinearMesh<2>, double, Brick> a(cell, layout, mesh);
+   Field<UniformRectilinearMesh<2>, double, Brick> b(cell, layout, mesh);
+ 
+   // Set up the initial conditions.
+   // All grid values should be zero except for the central value.
+   a = b = 0.0;
+   b(n/2,n/2) = 1000.0;
+ 
+   // Perform the simulation.
+   for (int k = 0; k < nuIterations; ++k) {
+     // Read from b.  Write to a.
+     a(I,J) = weight *
+       (b(I+1,J+1) + b(I+1,J  ) + b(I+1,J-1) +
+        b(I  ,J+1) + b(I  ,J  ) + b(I  ,J-1) +
+        b(I-1,J+1) + b(I-1,J  ) + b(I-1,J-1));
+ 
+     // Read from a.  Write to b.
+     b(I,J) = weight *
+       (a(I+1,J+1) + a(I+1,J  ) + a(I+1,J-1) +
+        a(I  ,J+1) + a(I  ,J  ) + a(I  ,J-1) +
+        a(I-1,J+1) + a(I-1,J  ) + a(I-1,J-1));
+   }
+ 
+   // Print out the final central value.
+   std::cout << b(n/2,n/2) << std::endl;
+ 
+   // The fields are automatically deallocated.
+ 
+   // Tell the Pooma library execution has finished.
+   Pooma::finalize();
+   return EXIT_SUCCESS;
+ }
Index: examples/Manual/Doof2d/include.mk
===================================================================
RCS file: include.mk
diff -N include.mk
*** /dev/null	Fri Mar 23 21:37:44 2001
--- include.mk	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,59 ----
+ # Generated by mm.pl: Mon Mar  9 13:58:39 MST 1998
+ # ACL:license
+ #  ----------------------------------------------------------------------
+ #  This software and ancillary information (herein called "SOFTWARE")
+ #  called POOMA (Parallel Object-Oriented Methods and Applications) is
+ #  made available under the terms described here.  The SOFTWARE has been
+ #  approved for release with associated LA-CC Number LA-CC-98-65.
+ #  
+ #  Unless otherwise indicated, this SOFTWARE has been authored by an
+ #  employee or employees of the University of California, operator of the
+ #  Los Alamos National Laboratory under Contract No. W-7405-ENG-36 with
+ #  the U.S. Department of Energy.  The U.S. Government has rights to use,
+ #  reproduce, and distribute this SOFTWARE. The public may copy, distribute,
+ #  prepare derivative works and publicly display this SOFTWARE without 
+ #  charge, provided that this Notice and any statement of authorship are 
+ #  reproduced on all copies.  Neither the Government nor the University 
+ #  makes any warranty, express or implied, or assumes any liability or 
+ #  responsibility for the use of this SOFTWARE.
+ #  
+ #  If SOFTWARE is modified to produce derivative works, such modified
+ #  SOFTWARE should be clearly marked, so as not to confuse it with the
+ #  version available from LANL.
+ #  
+ #  For more information about POOMA, send e-mail to pooma at acl.lanl.gov,
+ #  or visit the POOMA web page at http://www.acl.lanl.gov/pooma/.
+ #  ----------------------------------------------------------------------
+ # ACL:license
+ 
+ 
+ # Wrap make components from SHARED_ROOT and the current directory in the 
+ # proper order so that variables like ODIR have the correct directory-specific 
+ # value at the right moment.  See the included files for details of what they 
+ # are doing. This file should NOT be manually edited.
+ 
+ # Set NEXTDIR, THISDIR and DIR_LIST
+ include $(SHARED_ROOT)/include1.mk
+ 
+ # Include list of subdirectories to process
+ -include $(THISDIR)/subdir.mk
+ 
+ # Set ODIR, PROJECT_INCLUDES, UNIQUE
+ include $(SHARED_ROOT)/include2.mk
+ 
+ # Set list of object files, relative to ODIR
+ -include $(THISDIR)/objfile.mk
+ 
+ # Set rules for the ODIR directory 
+ include $(SHARED_ROOT)/compilerules.mk
+ 
+ # Remove current dir from DIR_LIST
+ DIR_LIST :=$(filter-out $(firstword $(DIR_LIST)), $(DIR_LIST))
+ 
+ 
+ # ACL:rcsinfo
+ #  ----------------------------------------------------------------------
+ #  $RCSfile: include.mk,v $   $Author: swhaney $
+ #  $Revision: 1.3 $   $Date: 2000/03/07 13:14:47 $
+ #  ----------------------------------------------------------------------
+ # ACL:rcsinfo
Index: examples/Manual/Doof2d/makefile
===================================================================
RCS file: makefile
diff -N makefile
*** /dev/null	Fri Mar 23 21:37:44 2001
--- makefile	Mon Dec  3 14:01:55 2001
***************
*** 0 ****
--- 1,96 ----
+ # Generated by mm.pl: Mon Mar  9 13:58:39 MST 1998
+ # ACL:license
+ #  ----------------------------------------------------------------------
+ #  This software and ancillary information (herein called "SOFTWARE")
+ #  called POOMA (Parallel Object-Oriented Methods and Applications) is
+ #  made available under the terms described here.  The SOFTWARE has been
+ #  approved for release with associated LA-CC Number LA-CC-98-65.
+ #  
+ #  Unless otherwise indicated, this SOFTWARE has been authored by an
+ #  employee or employees of the University of California, operator of the
+ #  Los Alamos National Laboratory under Contract No. W-7405-ENG-36 with
+ #  the U.S. Department of Energy.  The U.S. Government has rights to use,
+ #  reproduce, and distribute this SOFTWARE. The public may copy, distribute,
+ #  prepare derivative works and publicly display this SOFTWARE without 
+ #  charge, provided that this Notice and any statement of authorship are 
+ #  reproduced on all copies.  Neither the Government nor the University 
+ #  makes any warranty, express or implied, or assumes any liability or 
+ #  responsibility for the use of this SOFTWARE.
+ #  
+ #  If SOFTWARE is modified to produce derivative works, such modified
+ #  SOFTWARE should be clearly marked, so as not to confuse it with the
+ #  version available from LANL.
+ #  
+ #  For more information about POOMA, send e-mail to pooma at acl.lanl.gov,
+ #  or visit the POOMA web page at http://www.acl.lanl.gov/pooma/.
+ #  ----------------------------------------------------------------------
+ # ACL:license
+ 
+ # This file is user-editable
+ 
+ PROJECT_ROOT = $(shell cd ../../..; pwd)
+ include $(PROJECT_ROOT)/config/head.mk
+ 
+ PASS=APP
+ 
+ default:: Doof2d-C-element Doof2d-Array-element Doof2d-Array-parallel \
+ 	Doof2d-Array-stencil Doof2d-Array-distributed \
+ 	Doof2d-Field-parallel Doof2d-Field-distributed
+ 
+ .PHONY: Doof2d-C-element
+ 
+ Doof2d-C-element:: $(ODIR)/Doof2d-C-element
+ 
+ $(ODIR)/Doof2d-C-element: $(ODIR)/Doof2d-C-element.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Array-element
+ 
+ Doof2d-Array-element:: $(ODIR)/Doof2d-Array-element
+ 
+ $(ODIR)/Doof2d-Array-element: $(ODIR)/Doof2d-Array-element.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Array-parallel
+ 
+ Doof2d-Array-parallel:: $(ODIR)/Doof2d-Array-parallel
+ 
+ $(ODIR)/Doof2d-Array-parallel: $(ODIR)/Doof2d-Array-parallel.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Array-stencil
+ 
+ Doof2d-Array-stencil:: $(ODIR)/Doof2d-Array-stencil
+ 
+ $(ODIR)/Doof2d-Array-stencil: $(ODIR)/Doof2d-Array-stencil.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Array-distributed
+ 
+ Doof2d-Array-distributed:: $(ODIR)/Doof2d-Array-distributed
+ 
+ $(ODIR)/Doof2d-Array-distributed: $(ODIR)/Doof2d-Array-distributed.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Field-parallel
+ 
+ Doof2d-Field-parallel:: $(ODIR)/Doof2d-Field-parallel
+ 
+ $(ODIR)/Doof2d-Field-parallel: $(ODIR)/Doof2d-Field-parallel.o
+ 	$(LinkToSuite)
+ 
+ .PHONY: Doof2d-Field-distributed
+ 
+ Doof2d-Field-distributed:: $(ODIR)/Doof2d-Field-distributed
+ 
+ $(ODIR)/Doof2d-Field-distributed: $(ODIR)/Doof2d-Field-distributed.o
+ 	$(LinkToSuite)
+ 
+ include $(SHARED_ROOT)/tail.mk
+ 
+ # ACL:rcsinfo
+ #  ----------------------------------------------------------------------
+ #  $RCSfile: makefile,v $   $Author: oldham $
+ #  $Revision: 1.1 $   $Date: 2000/07/21 21:34:44 $
+ #  ----------------------------------------------------------------------
+ # ACL:rcsinfo


More information about the pooma-dev mailing list