Difference between revisions of "Main Page/PIO"
(57 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | This is the document page for parallel I/O library developed | + | This is the document page for parallel I/O library developed for [http://www.mcs.anl.gov/~mmin/nekcem.html NekCEM]. |
== Background == | == Background == | ||
Line 5: | Line 5: | ||
:Binary (used for production, compact size), or ASCII (used for debugging, human-readable) | :Binary (used for production, compact size), or ASCII (used for debugging, human-readable) | ||
− | == Usage Introduction | + | == Usage Introduction == |
− | + | Users can use #1,2,3,4,5,6,7,8 in .rea file for a specific example, by parameter 81. <br> | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | Several '''advanced parallel I/O''' algorithms based on MPI-IO library were developed. <br> | |
+ | The analysis on the performance for those approaches are detailed in ''Parallel I/O | ||
+ | performance for application-level checkpointing on the Blue Gene/P, by Jing Fu et. al.'' [http://www.mcs.anl.gov/~mmin] | ||
− | param( | + | proc=processors |
+ | '''param(81) = 4''': collective IO for N proc to 1 file ---> mpi-binary-N1-xxx.vtk | ||
+ | '''param(81) = 5''': collective IO for N proc to multiple M-files ---> mpi-binary-NM-xxx.vtk | ||
+ | '''param(81) = 6''': reduced-blocking IO for N proc to 1 file with M writers ---> mpi-binary-NM1-xxx.vtk | ||
+ | '''param(81) = 7''': reduced-blocking IO for N proc to 1 file with M writers ---> mpi-ascii-NM1-xxx.vtk | ||
+ | '''param(81) = 8''': reduced-blocking IO for N proc to multiple M-files with M writers ---> mpi-binary-NMM-xxx.vtk | ||
− | param( | + | '''Note''' that param(82) and param(83) need to be set correctly in *.rea file. |
+ | '''param(80)''' = ''exact'' total number of fields to be written | ||
+ | '''param(82)''' = number of output files | ||
+ | '''param(83)''' = frequency of restart output files, 0 (no restart output), #nn (iostep*nn) | ||
+ | '''param(84)''' = invoked with dump_number from the name of the restarting output file | ||
− | |||
− | + | ---- | |
− | + | For '''traditional I/O''' approaches based on one file per processor, using ''old libraries'', one can use | |
− | |||
− | |||
− | |||
− | + | '''param(81) = 2''': use Fortran I/O library (ASCII, VTK format) ---> ascii-xxx.vtk | |
− | + | '''param(81) = 3''': use C-POSIX I/O libraries (binary, VTK format) ---> binary-xxx.vtk | |
+ | '''param(81) = 1''': use nek5000's old output format (ASCII) -> xxx.fld | ||
+ | == Visualization == | ||
− | == | + | The output files with param(81)=2,3,4,5,6,7,8 can be visualized with '''ParaView''' and '''VisIt'''. <br> |
+ | The output with param(81)=1 can be visualized with nek5000 visualization tool, '''postx''' <br> |
Latest revision as of 22:36, 16 December 2011
This is the document page for parallel I/O library developed for NekCEM.
Background
- File Format
- Binary (used for production, compact size), or ASCII (used for debugging, human-readable)
Usage Introduction
Users can use #1,2,3,4,5,6,7,8 in .rea file for a specific example, by parameter 81.
Several advanced parallel I/O algorithms based on MPI-IO library were developed.
The analysis on the performance for those approaches are detailed in Parallel I/O
performance for application-level checkpointing on the Blue Gene/P, by Jing Fu et. al. [1]
proc=processors param(81) = 4: collective IO for N proc to 1 file ---> mpi-binary-N1-xxx.vtk param(81) = 5: collective IO for N proc to multiple M-files ---> mpi-binary-NM-xxx.vtk param(81) = 6: reduced-blocking IO for N proc to 1 file with M writers ---> mpi-binary-NM1-xxx.vtk param(81) = 7: reduced-blocking IO for N proc to 1 file with M writers ---> mpi-ascii-NM1-xxx.vtk param(81) = 8: reduced-blocking IO for N proc to multiple M-files with M writers ---> mpi-binary-NMM-xxx.vtk
Note that param(82) and param(83) need to be set correctly in *.rea file.
param(80) = exact total number of fields to be written param(82) = number of output files param(83) = frequency of restart output files, 0 (no restart output), #nn (iostep*nn) param(84) = invoked with dump_number from the name of the restarting output file
For traditional I/O approaches based on one file per processor, using old libraries, one can use
param(81) = 2: use Fortran I/O library (ASCII, VTK format) ---> ascii-xxx.vtk param(81) = 3: use C-POSIX I/O libraries (binary, VTK format) ---> binary-xxx.vtk
param(81) = 1: use nek5000's old output format (ASCII) -> xxx.fld
Visualization
The output files with param(81)=2,3,4,5,6,7,8 can be visualized with ParaView and VisIt.
The output with param(81)=1 can be visualized with nek5000 visualization tool, postx