Index: /issm/trunk/externalpackages/petsc/ex-petsc.txt =================================================================== --- /issm/trunk/externalpackages/petsc/ex-petsc.txt (revision 3978) +++ /issm/trunk/externalpackages/petsc/ex-petsc.txt (revision 3978) @@ -0,0 +1,437 @@ + + PETSc Tutorial Exercises + + Intended for use with version 2.0.29 of PETSc + + +*************** +Getting Started +*************** + + - Locate the PETSc on-line documentation, available via + http://www.mcs.anl.gov/petsc/docs + In particular, you may want to reference the on-line users manual + and manual pages. + + - Set the following environmental variables: + PETSC_DIR - location of PETSc installation + PETSC_ARCH - machine on which you will use PETSc + e.g., setenv PETSC_DIR /users/gropp/petsc-2.0.29 + setenv PETSC_ARCH alpha + At some sites, there may be commands such as module or soft to set these. + + - Explore some of the following example exercises. First, copy + an example program and its corresponding makefile from the PETSc + distribution into a working directory. For example, to use the + "hello world" example listed below, do the following: + % mkdir petsc_examples + % cd petsc_examplesi + % mkdir hello_world + % cd hello_world + % cp $PETSC_DIR/src/sys/examples/tutorials/ex2.c . + % cp $PETSC_DIR/src/sys/examples/tutorials/makefile . + Note that each of the examples categories should be placed in + a separate working directory so that the makefile commands will + function properly. + + - Compile an example program: + debugging version: + make BOPT=g , e.g., + make BOPT=g ex2 + optimized version: This is the letter O, not the digit zero + make BOPT=O , e.g., + make BOPT=O ex2 + + - Run an example program: + mpirun -np , e.g., + mpirun -np 2 ex2 + +****************** +Beginner Exercises +****************** + +- - - - - - +Hello World +- - - - - - + + * Objective: To run a simple program and learn the makefile and run commands + Program: $PETSC_DIR/src/sys/examples/tutorials/ex2.c + Compilation commands: + debugging version: make BOPT=g ex2 + optimized version: make BOPT=O ex2 + Runtime commands: + mpirun -np 1 ex2 + mpirun -np 2 ex2 + mpirun -np 4 ex2 + +- - - - - - - +Vectors (Vec) +- - - - - - - + + * Objectives: To demonstrate basic vector capabilities. + To demonstrate creation/destruction of PETSc objects. + Programs: $PETSC_DIR/src/vec/examples/tutorials/ex1.c + or $PETSC_DIR/src/vec/examples/tutorials/ex1f.F + Runtime commands: + mpirun -np 2 ex1 + or + mpirun -np 2 ex1f + + * Objective: To demonstrate parallel vector layout + Programs: $PETSC_DIR/src/vec/examples/tutorials/ex3.c + or $PETSC_DIR/src/vec/examples/tutorials/ex3f.F + Runtime commands: + mpirun -np 2 ex3 -draw_pause -1 + or + mpirun -np 2 ex3f -draw_pause -1 + where + -draw_pause -1 : click the RIGHT mouse button to + advance the program + +- - - - - - - - +Matrices (Mat) +- - - - - - - - + + * Objective: To demonstrate matrix assembly + Program: $PETSC_DIR/src/sles/examples/tutorials/ex3.c + Runtime commands: + mpirun -np 2 ex3 -mat_view + and + mpirun -np 2 ex3 -mat_view_draw -draw_pause -1 + where + -mat_view : print matrix elements to screen + -mat_view_draw : draw the matrix sparsity structure + + Comments: + Note that this matrix assembly is done by 4x4 blocks, + with each process contributing its own local parts; see + calls to MatSetValues(). + + In this example, you may need to move one of the windows that + pops up aside so that you can right-click the one underneath + to continue the program's execution. + +- - - - - - - - - - - +Linear Solvers (SLES) +- - - - - - - - - - - + +SLES provides an easy-to-use interface to the combination of a +Krylov subspace iterative solver (KSP) and a preconditioner (PC). +See the SLES, KSP, and PC manual pages as well as the "Linear +Equations Solvers" chapter of the PETSc users manual for details. + + * Objective: To demonstrate basic linear solver use + Program: $PETSC_DIR/src/sles/examples/tutorials/ex23.c + Runtime command: + mpirun -np 2 ex23 -ksp_monitor + where + -ksp_monitor : print residual norm at each iteration + Comments: + Note that the program calls SLESView(), which displays + information about the particular solvers used at runtime. + + * Objective: To demonstrate setting different preconditioners and + Krylov methods at runtime + Program: $PETSC_DIR/src/sles/examples/tutorials/ex23.c + Runtime command: + mpirun -np 2 ex23 -ksp_monitor -pc_type asm -ksp_type tfqmr -optionsleft + where + -ksp_type tfqmr : set Krylov method to transpose-free QMR + -pc_type asm : set preconditioner to additive Schwarz + -optionsleft : print information about the options specified at runtime + Comments: + Use the -help option for a complete list of solver options. + + * Objective: To produce a summary of program performance + Program: $PETSC_DIR/src/sles/examples/tutorials/ex23.c + Runtime command: + mpirun -np 4 ex23 -n 1259 -log_summary + where + -n : set the problem dimension + Comments: + The performance summary output is wide, so use an xterm with + 120 columns; either resize one or use "xterm -geometry 120x24". + + See the chapter "Hints for Performance Tuning" in the PETSc users + manual for help in interpreting this output. + + Try using additional solver options as well, and note the + effects on performance. For example, + -ksp_rtol 1.e-10 : set convergence tolerance + Use the -help option for a complete list of solver options. + +- - - - - - - - - - - - +Nonlinear Solvers (SNES) +- - - - - - - - - - - - + + * Objective: To demonstrate basic nonlinear solver use + Programs: $PETSC_DIR/src/snes/examples/tutorials/ex5.c + or $PETSC_DIR/src/snes/examples/tutorials/ex5f.F and ex5f.h + or $PETSC_DIR/src/snes/examples/tutorials/ex5f90.F + Runtime commands: + mpirun -np 3 ex5 -mx 10 -my 12 -snes_monitor -snes_view + or + mpirun -np 3 ex5f -mx 10 -my 12 -snes_monitor -snes_view + or + mpirun -np 3 ex5f90 -mx 10 -my 12 -snes_monitor -snes_view + where + -snes_monitor : print residual norm at each iteration + -snes_view : print information about the particular + nonlinear solvers used at runtime + -mx -my : set mesh dimensions + Comments: + By default a Netwon line search method is used. + + * Objective: To demonstrate setting different nonlinear solvers at runtime + Programs: $PETSC_DIR/src/snes/examples/tutorials/ex5.c + or $PETSC_DIR/src/snes/examples/tutorials/ex5f.F and ex5f.h + or $PETSC_DIR/src/snes/examples/tutorials/ex5f90.F + Runtime command: + mpirun -np 3 ex5 -mx 10 -my 12 -snes_monitor -snes_view \ + -snes_type tr -optionsleft + (or similarly for ex5f and ex5f90) + where + -snes_type tr : set nonlinear solver to a Newton trust region method + -optionsleft : print information about the options specified at runtime + Comments: + Use the -help option for a complete list of solver options. + +- - - - - - - - - - - - - +Timestepping Solvers (TS) +- - - - - - - - - - - - - + + * Objective: To demonstrate basic timestepping solver use + Programs: $PETSC_DIR/src/ts/examples/tutorials/ex2.c and util2.c + or $PETSC_DIR/src/ts/examples/tutorials/ex2f.F, util2.c, and ex2f.h + Runtime command: + mpirun -np 2 ex2 -ts_view + or + mpirun -np 2 ex2f -ts_view + where + -ts_view : print information about the particular + timestepping solvers used at runtime + Comments: + The backward Euler method is set in this code by a call to + TSSetType(). This example runs for 1000 time steps. + + * Objective: To demonstrate setting different timestepping solvers at runtime + Programs: $PETSC_DIR/src/ts/examples/tutorials/ex2.c and util2.c + or $PETSC_DIR/src/ts/examples/tutorials/ex2f.F, util2.c, and ex2f.h + Runtime command: + mpirun -np 2 ex2 -ts_view -ts_type euler + or + mpirun -np 2 ex2f -ts_view -ts_type euler + where + -ts_type euler : set timestepping solver to the Euler method + Comments: + Use the -help option for a complete list of solver options. + +********************** +Intermediate Exercises +********************** + +- - - - - - - - - - - - - - - - - - +Mesh Management (DA and VecScatter) +- - - - - - - - - - - - - - - - - - + + * Objective: To demonstrate use of distributed arrays (DA) to + manage a parallel structured mesh computation. + Program: $PETSC_DIR/src/snes/examples/tutorials/ex5.c + Runtime command: + mpirun -np 4 ex5 -mx 10 -my 11 -da_view_draw -snes_monitor \ + -draw_pause -1 + where + -da_view_draw : draw DA configuration + + * Objective: To demonstrate use of vector scatters (VecScatter) + to manage a parallel unstructured mesh computation. + Program: $PETSC_DIR/src/snes/examples/tutorials/ex10d/ex10.c and adj.in + Runtime command: + mpirun -np 2 ex10 -snes_monitor + +- - - - - - - - - - - - - - - - - - + Evaluating and Tuning Performance +- - - - - - - - - - - - - - - - - - + +See the chapter "Hints for Performance Tuning" in the PETSc users manual. + + * Objective: To demonstrate verbose logging information + Program: $PETSC_DIR/src/sles/examples/tutorials/ex23.c + (and any other programs as well) + Runtime command: + mpirun -np 2 ex23 -log_info + where + -log_info : print verbose logging information + Comments: + Note that this command prints information about matrix assembly, + such as the number of dynamic memory allocations used, maximum + number of nonzeros per row, etc. This information can help + with tuning the matrix assembly process. + + +****************************************************** +Advanced Exercises +****************************************************** + + * Objective: Demonstrate various multigrid, additive Schwarz, + and Neumann-Neumann preconditioner options for + several test problems. + +- - - - - - - - - - - +Linear Solvers (SLES) +- - - - - - - - - - - + + * Objective: 3D Laplacian on a structured mesh + Program: $PETSC_DIR/src/sles/examples/tutorials/ex22.c + + Multigrid: + mpirun -np 1 ex22 -sles_view -ksp_monitor + (Note: the default smoother is ILU(0)) + mpirun -np 1 ex22 -sles_view -ksp_monitor \ + -mg_levels_pc_type sor (use sor as a smoother) + mpirun -np 4 ex22 -sles_view -ksp_monitor \ + -mg_levels_pc_type sor -mg_levels_pc_sor_local_symmetric + + Try some additional multigrid options: + - Use a different number of levels: + -damg_nlevels + - Use a W-cycle instead of the default V-cycle: + -pc_mg_cycles 2 + - Use a different scheme: + -pc_mg_type + - Use 2 pre and 2 post smoothing steps: + -pc_mg_smoothup 2 -pc_mg_smoothdown 2 + Also try different combinations of these. + + One-level additive Schwarz: (one domain per processor) + mpirun -np 1 ex22 -sles_view -ksp_monitor -pc_type asm + (Note: This has one subdomain, so is the same as ILU(0), the + default subdomain solver) + mpirun -np 4 ex22 -sles_view -ksp_monitor -pc_type asm + (Note: default overlap of 1) + mpirun -np 4 ex22 -sles_view -ksp_monitor -pc_type asm -sub_pc_type lu + (Solve the subproblems exactly with LU) + mpirun -np 4 ex22 -sles_view -ksp_monitor -pc_type asm -pc_asm_overlap 2 + + Two-level additive Schwarz: (grid ratio of 32 between levels) + mpirun -np 4 ex22 -sles_view -ksp_monitor -damg_nlevels 2 -damg_ratio 32 + + * Objective: 2D Laplacian on an unstructured grid + Program: $PETSC_DIR/src/contrib/oberman/laplacian_q1/* (copy + everything in this directory) + + Compilation: + make BOPT=g main (debugging version) + or + make BOPT=g main (optimized version) + + Default solver: block Jacobi with ILU(0) on each block + mpirun -np 4 main -f gr10 (also try using larger grids: gr22, gr44, gr99) + + One-level additive Schwarz: + mpirun -np 4 main -f gr10 -pc_type asm + (Note: default is Restricted Additive Schwarz (RASM)) + mpirun -np 4 main -f gr10 -pc_type asm -pc_asm_type basic + (This is "standard" additive Schwarz, where the subdomain + restriction and interpolation are the same.) + Try varying the overlap, subdomain solver, etc. + + Mandel's balancing Neumann-Neumann preconditioner: + mpirun -np 4 main -ksp_monitor -f gr10 -pc_type nn -mat_type is + mpirun -np 8 main -ksp_monitor -f gr33n -pc_type nn -mat_type is + (Note: This mesh has Dirichlet boundary conditions only on the left edge) + mpirun -np 8 main -ksp_monitor -f gr33n -pc_type nn -mat_type is \ + -turn_off_first_balancing -turn_off_second_balancing + (Run with no coarse problem; note that this requires many + more iterations.) + + +- - - - - - - - - - - - +Nonlinear Solvers (SNES) +- - - - - - - - - - - - + + * Objective: Scalar, radiative transport PDE on a structured mesh + Program: $PETSC_DIR/src/snes/examples/tutorials/ex18.c + + mpirun -np 2 ex18 -snes_view -snes_monitor + + The problem is run twice to preload the memory. + + Run the program with the different options suggested above + for multigrid and additive Schwarz. + + * Objective: Multicomponent, driven cavity problem (with 4 degrees + of freedom per node) on a structured mesh + Program: $PETSC_DIR/src/snes/examples/tutorials/ex19.c + + mpirun -np 2 ex19 -snes_view -snes_monitor + + Run the program with the different options suggested above + for multigrid and additive Schwarz. + + +****************************************************** +Interfacing between PETSc and other software packages +****************************************************** + +See slide #7 for web page pointers to these software packages. + +To use one of these external tools with PETSc, the tool must have been +enabled when the local PETSc installation was built (see the file +$PETSC_DIR/bmake/<$PETSC_ARCH>/base.site). The following tools +[may/may not] be available on your machine. + +- - - - - - - - - - - - - - - - - - - - - - - - - + SPAI - sparse approximate inverse preconditioner +- - - - - - - - - - - - - - - - - - - - - - - - - + + Program: $PETSC_DIR/src/sles/examples/tutorials/ex23.c + Runtime commands: + mpirun -np 1 ex23 -ksp_monitor -pc_type spai + Comments: + Use the -help option for a complete list of SPAI options. + +- - - - - - - - - - - - - - - - - - + ParMETIS - parallel partitioner +- - - - - - - - - - - - - - - - - - + + Program: $PETSC_DIR/src/dm/ao/examples/tutorials/ex2.c + Runtime command: + mpirun -np 2 ex2 -mat_partitioning_type parmetis + +- - - - - - - - - - - - + PVODE - ODE integrator +- - - - - - - - - - - - + + Programs: $PETSC_DIR/src/ts/examples/tutorials/ex2.c + or $PETSC_DIR/src/ts/examples/tutorials/ex2f.F + Runtime commands: + mpirun -np 2 ex2 -ts_type pvode + or + mpirun -np 2 ex2f -ts_type pvode + +- - - - - - - - - - - - - - - - - - - - + Overture - composite mesh PDE package +- - - - - - - - - - - - - - - - - - - - + + Please speak with us if you are interested in the Overture/PETSc interface. + +- - - - - - - - - - - + SAMRAI - AMR package +- - - - - - - - - - - + + Please speak with us if you are interested in the SAMRAI/PETSc interface. + +- - - - - - - - - - - - - - - - - - - - - - + TAO - large-scale optimization software +- - - - - - - - - - - - - - - - - - - - - - + + See the TAO installation (TAO_DIR = /u3/sbenson/tao-0.0.2) + and tutorial, including programs: + unconstrained minimization: + $TAO_DIR/src/unconstrained/examples/tutorials/eptorsion2.c + $TAO_DIR/src/unconstrained/examples/tutorials/eptorsion2f.F + bound constrained minimization: + $TAO_DIR/src/bound/examples/tutorials/plate2.c + $TAO_DIR/src/bound/examples/tutorials/plate2f.F +