This directory contains various useful utilities

bfort is a sun executable that generates wrappers for C routines to be called
from Fortran.

/*D
    mpirun - Run mpi programs

Description:
"mpirun" is a shell script that attempts to hide the differences in
starting jobs for various devices from the user. Mpirun attempts to
determine what kind of machine it is running on and start the required
number of jobs on that machine. On workstation clusters, if you are
not using Chameleon, you must supply a file that lists the different
machines that mpirun can use to run remote jobs or specify this file
every time you run mpirun with the -machine file option. The default
file is in util/machines/machines.<arch>.

mpirun typically works like this
.vb
 mpirun -np <number of processes> <program name and arguments>
.ve

If mpirun cannot determine what kind of machine you are on, and it 
is supported by the mpi implementation, you can the -machine
and -arch options to tell it what kind of machine you are running
on. The current valid values for machine are

.vb
          chameleon (including chameleon/pvm, chameleon/p4, etc...)
          meiko     (the meiko device on the meiko)
          paragon   (the ch_nx device on a paragon not running NQS)
          p4        (the ch_p4 device on a workstation cluster)
          ibmspx    (ch_eui for IBM SP2)
          anlspx    (ch_eui for ANLs SPx)
          ksr       (ch_p4 for KSR 1 and 2)
          sgi_mp    (ch_shmem for SGI multiprocessors)
          cray_t3d  (t3d for Cray T3D)
          smp       (ch_shmem for SMPs)
          execer    (a custom script for starting ch_p4 programs
	             without using a procgroup file. This script
                     currently does not work well with interactive
    	             jobs)
.ve
You should only have to specify mr_arch if mpirun does not recognize
your machine, the default value is wrong, or you are using the p4 or
execer devices.  The full list of options is

Parameters:
The options for mpirun must come before the program you want to run and
must be spelled out completely (no abreviations).  Unrecognized options 
will be silently ignored.

mpirun [mpirun_options...] <progname> [options...]

+   -arch <architecture> -
            specify the architecture (must have matching machines.<arch>
            file in ${MPIR_HOME}/util/machines) if using the execer
.    -h      - This help
.    -machine <machine name> -
            use startup procedure for <machine name>
.    -machinefile <machine\-file name> -
            Take the list of possible machines to run on from the
            file <machine-file name>
.    -np <np> -
            specify the number of processors to run on
.    -nolocal -
            do not run on the local machine (only works for 
            p4 and ch_p4 jobs)
.    -stdin filename -
            Use filename as the standard input for the program.  This
            is needed for programs that must be run as batch jobs, such
            as some IBM SP systems and Intel Paragons using NQS (see 
            -paragontype below).
.    -t      - Testing - do not actually run, just print what would be
            executed
.    -v      - Verbose - throw in some comments
.    -dbx    - Start the first process under dbx where possible
.    -gdb    - Start the first process under gdb where possible
             (on the Meiko, selecting either -dbx or -gdb starts prun
             under totalview instead)
.    -xxgdb  - Start the first process under xxgdb where possible (-xdbx
            does not work)
-    -tv     - Start under totalview

      Special Options for NEC - CENJU-3:

+    -batch  - Excecute program as a batch job (using cjbr)

.    -stdout filename -
            Use filename as the standard output for the program.
-    -stderr filename -
            Use filename as the standard error  for the program.

     Special Options for Nexus device:
  
+    -nexuspg filename -
	    Use the given Nexus startup file instead of creating one.
	    Overrides -np and -nolocal, selects -leave_pg.
    
-    -nexusdb filename -
	    Use the given Nexus resource database.

    Special Options for Workstation Clusters:


+    -e     - Use execer to start the program on workstation
            clusters
.    -pg    - Use a procgroup file to start the p4 programs, not execer
            (default)
.    -leave_pg -
            Do not delete the P4 procgroup file after running
.    -p4pg filename -
            Use the given p4 procgroup file instead of creating one.
            Overrides -np and -nolocal, selects -leave_pg.
.    -tcppg filename -
            Use the given tcp procgroup file instead of creating one.
            Overrides -np and -nolocal, selects -leave_pg.
-    -p4ssport num -
            Use the p4 secure server with port number num to start the
            programs.  If num is 0, use the value of the 
            environment variable MPI_P4SSPORT.  Using the server can
            speed up process startup.  If MPI_USEP4SSPORT as well as
            MPI_P4SSPORT are set, then that has the effect of giving
            mpirun the -p4ssport 0 parameters.

    Special Options for Batch Environments:

+    -mvhome - Move the executable to the home directory.  This 
            is needed when all file systems are not cross-mounted.
            Currently only used by anlspx
.    -mvback files -
            Move the indicated files back to the current directory.
            Needed only when using -mvhome; has no effect otherwise.
.    -maxtime min -
            Maximum job run time in minutes.  Currently used only
            by anlspx.  Default value is 15 minutes
.    -nopoll - Do not use a polling-mode communication.
            Available only on IBM SPx.
.    -mem value -
	   This is the per node memory request (in Mbytes).  Needed for some
           CM-5s.
-    -cpu time -
	   This is the the hard cpu limit used for some CM-5s in
	   minutes.

    Special Options for IBM SP2:

.    -cac name -
            CAC for ANL scheduler.  Currently used only by anlspx.
            If not provided will choose some valid CAC.

    Special Options for Intel Paragon:

+    -paragontype name -
           Selects one of default, mkpart, NQS, depending on how you want
           to submit jobs to a Paragon.

.    -paragonname name -
           Remote shells to name to run the job (using the -sz method) on 
           a Paragon.

-    -paragonpn name -
           Name of partition to run on in a Paragon (using the -pn name 
           command-line argument)

Return value:

On exit, mpirun returns a status of zero unless mpirun detected a problem, in
which case it returns a non-zero status (currently, all are one, but this 
may change in the future).

Specifying Heterogeneous Systems:

Multiple architectures may be handled by giving multiple '-arch' and '-np' 
arguments.  For example, to run a program on 2 sun4s and 3 rs6000s, with 
the local machine being a sun4, use
.vb
    mpirun -arch sun4 -np 2 -arch rs6000 -np 3 program
.ve
This assumes that program will run on both architectures.  If different
executables are needed (as in this case), the string '%a' will be replaced
with the arch name. For example, if the programs are 'program.sun4' and
'program.rs6000', then the command is
.vb
    mpirun -arch sun4 -np 2 -arch rs6000 -np 3 program.%a
.ve
If instead the execuables are in different directories; for example, 
'/tmp/me/sun4' and '/tmp/me/rs6000', then the command is
.vb
    mpirun -arch sun4 -np 2 -arch rs6000 -np 3 /tmp/me/%a/program
.ve
It is important to specify the architecture with '-arch' `before` specifying
the number of processors.  Also, the `first` '-arch' command must refer to the
processor on which the job will be started.  Specifically, if '-nolocal' is 
`not` specified, then the first -arch must refer to the processor from which
mpirun is running.

(You must have 'machines.<arch>' files for each arch that you use in the
'util/machines' directory.)

Another approach that may be used the the 'ch_p4' device is to create a
'procgroup' file directly.  See the MPICH Users Guide for more information.

D*/

/*D
    mpicc - Compiles and links MPI programs written in C

    Description:
    This command can be used to compile and link MPI programs written in
    C.  It provides the options and any special libraries that are
    needed to compile and link MPI programs.  

    It is important to use this command (or a 'Makefile' processed with
    'mpireconfig') particularly when linking programs, as it provides 
    the necessary libraries.  It can also simplify the use of the MPE 
    profiling libraries, through the use of the '-mpilog', '-mpitrace', 
    and '-mpianim' commands.

    Command line arguments:
+  -mpilog    - Build version that generate MPE log files
.  -mpitrace  - Build version that generates traces
.  -mpianim   - Build version that generates real-time
             animation
.  -show      - Show the commands that would be used without
             runnning them
.  -help      - Give short help
.  -echo      - Show exactly what this program is doing.
             This option should normally not be used.
-  others     - are passed to the compiler or linker.  For example, '-c'
    causes files to be compiled, '-g' selects compilation with debugging
    on most systems, and '-o name' causes linking with the output
    executable given the name 'name'.

    Environment Variables:
    The environment variables 'MPICH_CC' and 'MPICH_CLINKER' may be used
    to select different C compiler and linker.  Note that since
    MPICH is built with a particular C and Fortran compiler, change the
    compilers used can cause problems.  Use this only if you could intermix
    code compiled with the different compilers.

    Examples:
To compile a single file 'foo.c', use
.vb
   mpicc -c foo.c 
.ve

To link the output and make an executable, use
.vb
   mpicc -o foo foo.o
.ve
Combining compilation and linking in a single command
.vb
   mpicc -o foo foo.c
.ve
is a convenient way to build simple programs.

.seealso: mpif77, mpireconfig
D*/

/*D
    mpif77 - Compiles and links MPI programs written in Fortran 77

    Description:
    This command can be used to compile and link MPI programs written in
    Fortran 77.  It provides the options and any special libraries that are
    needed to compile and link MPI programs.  

    It is important to use this command (or a 'Makefile' processed with
    'mpireconfig') particularly when linking programs, as it provides 
    the necessary libraries.  It can also simplify the use of the MPE 
    profiling libraries, through the use of the '-mpilog', '-mpitrace', 
    and '-mpianim' commands.

    Command line arguments:
+  -mpilog    - Build version that generate MPE log files
.  -mpitrace  - Build version that generates traces
.  -mpianim   - Build version that generates real-time
             animation
.  -show      - Show the commands that would be used without
             runnning them
.  -help      - Give short help
.  -echo      - Show exactly what this program is doing.
             This option should normally not be used.
-  others     - are passed to the compiler or linker.  For example, '-c'
    causes files to be compiled, '-g' selects compilation with debugging
    on most systems, and '-o name' causes linking with the output
    executable given the name 'name'.

    Environment Variables:
    The environment variables 'MPICH_F77' and 'MPICH_F77LINKER' may be used
    to select different Fortran 77 compiler and linker.  Note that since
    MPICH is built with a particular C and Fortran compiler, change the
    compilers used can cause problems.  Use this only if you could intermix
    code compiled with the different compilers.

    Examples:

To compile a single file 'foo.f', use
.vb
   mpif77 -c foo.f
.ve

To link the output and make an executable, use
.vb
   mpif77 -o foo foo.o
.ve
Combining compilation and linking in a single command
.vb
   mpif77 -o foo foo.f
.ve
is a convenient way to build simple programs.

.seealso: mpicc, mpireconfig
D*/

/*D
    mpif90 - Compiles and links MPI programs written in Fortran 90

    Description:
    This command can be used to compile and link MPI programs written in
    Fortran 90.  It provides the options and any special libraries that are
    needed to compile and link MPI programs.  

    This command is still under development, as is support in MPICH
    for Fortran 90.  

    It is important to use this command (or a 'Makefile' processed with
    'mpireconfig') particularly when linking programs, as it provides 
    the necessary libraries.  It can also simplify the use of the MPE 
    profiling libraries, through the use of the '-mpilog', '-mpitrace', 
    and '-mpianim' commands.

    Command line arguments:
+  -mpilog    - Build version that generate MPE log files
.  -mpitrace  - Build version that generates traces
.  -mpianim   - Build version that generates real-time
             animation
.  -show      - Show the commands that would be used without
             runnning them
.  -help      - Give short help
.  -echo      - Show exactly what this program is doing.
             This option should normally not be used.
-  others     - are passed to the compiler or linker.  For example, '-c'
    causes files to be compiled, '-g' selects compilation with debugging
    on most systems, and '-o name' causes linking with the output
    executable given the name 'name'.

    Environment Variables:
    The environment variables 'MPICH_F90' and 'MPICH_F90LINKER' may be used
    to select different Fortran compiler and linker.  Note that since
    MPICH is built with a particular C and Fortran compiler, change the
    compilers used can cause problems.  Use this only if you could intermix
    code compiled with the different compilers.

    Examples:

To compile a single file 'foo.f', use
.vb
   mpif90 -c foo.f
.ve

To link the output and make an executable, use
.vb
   mpif90 -o foo foo.o
.ve
Combining compilation and linking in a single command
.vb
   mpif90 -o foo foo.f
.ve
is a convenient way to build simple programs.

.seealso: mpicc, mpif77, mpiCC, mpireconfig
D*/

/*D
    mpiCC - Compiles and links MPI programs written in C++

    Description:
    This command can be used to compile and link MPI programs written in
    C++.  It provides the options and any special libraries that are
    needed to compile and link MPI programs.  

    It is important to use this command (or a 'Makefile' processed with
    'mpireconfig') particularly when linking programs, as it provides 
    the necessary libraries.  It can also simplify the use of the MPE 
    profiling libraries, through the use of the '-mpilog', '-mpitrace', 
    and '-mpianim' commands.

    mpiCC cannot be used to compile C programs.  Use mpicc for C programs.

    Command line arguments:
+  -mpilog    - Build version that generate MPE log files
.  -mpitrace  - Build version that generates traces
.  -mpianim   - Build version that generates real-time
             animation
.  -show      - Show the commands that would be used without
             runnning them
.  -help      - Give short help
.  -echo      - Show exactly what this program is doing.
             This option should normally not be used.
-  others     - are passed to the compiler or linker.  For example, '-c'
    causes files to be compiled, '-g' selects compilation with debugging
    on most systems, and '-o name' causes linking with the output
    executable given the name 'name'.

    Environment Variables:
    The environment variables 'MPICH_CCC' and 'MPICH_CCLINKER' may be used
    to select different C++ compiler and linker.  Note that since
    MPICH is built with a particular C++ and C compiler, change the
    compilers used can cause problems.  Use this only if you could intermix
    code compiled with the different compilers.

    Examples:
To compile a single file 'foo.c', use
.vb
   mpicc -c foo.c 
.ve

To link the output and make an executable, use
.vb
   mpicc -o foo foo.o
.ve
Combining compilation and linking in a single command
.vb
   mpicc -o foo foo.c
.ve
is a convenient way to build simple programs.

.seealso: mpif77, mpireconfig
D*/

/*D
   mpireconfig - Constucts makefiles from template files

   Description:
   The 'mpireconfig' command takes a template file and creates a 
   file that may be given to the 'make' command by replacing variable
   with values appropriate for a particular MPICH configuration.

   Syntax:

   mpireconfig filename

   Command Line Arguments:
.  filename   - Name of file to be generated.  The file 'filename.in'
   must exist.

.seealso: mpicc, mpif77, mpif90, mpiCC  
 D*/

/*D
   tstmachines - Tests the availability of remote machines for runing MPI
    programs

   Description:
   This script runs some simple tests to test that the environment can 
   support the 'ch_p4' device.  

D*/

/*D
   chp4_servs - Starts the P4 secure servers on machines known to MPICH

   Description:    
   This script can be used `when there is a working remote shell program`
   to start the secure servers that the 'ch_p4' device can use to speed
   up process startup.
D*/

/*D
   mpiman - Starts reader for MPI man pages in Unix man or Web HTML format

   Description:
   The 'mpiman' command gives access to the MPI manual pages in both
   UNIX man and Web HTML format.  By default, 'mpiman' uses xman, the
   X Window System manual browser to read the man pages.  By providing
   a command-line argument to 'mpiman', you can choose a different browser.
   The Web HTML format provides hot links between related routines and
   with the MPI standard.

   Command Line Arguments:
+  -xmosaic - Use the 'xmosaic' Web browser.
.  -mosaic  - Use the 'mosaic' Web browser.  'xmosaic' and 'mosaic' are often
    the same program
.  -netscape - Use the 'netscape' Web browser.
.  -xman     - Use the X Window System xman Unix man browser
-  -man      - Use the man program (e.g., 'mpiman -man MPI_Send')

D*/

/*D
    cleanipcs - Removes System V IPCs held by the user

    Description: 
    This command frees the shared memory segments and semaphores help by
    the user that calls this command.  This command is normally not 
    needed, but if MPI programs exit abnormally, the program may be unable
    to free the System V IPCs that it held (this is a feature of System V
    IPCs).  In that case, cleanipcs can be used to recover the IPCs that
    are no longer needed.

    Note that this command releases `all` IPCs held by the user.  This is
    the correct behavior for most users, but may cause problems for users
    of other programs or systems that rely on the persistence of System V 
    IPCs.

    Command Line Arguments:
+   -show - Show the commands that would be executed, but do not actually 
            execute them
-   -echo - Show all commands executed by this script
D*/
