parIO - parallel IO for Fortran MPI applications
Parallel IO has to be robust, fast and still easy to use - this can become a difficult task, especially in highly parallel environments. HDF5 has proven to be the first choice for parallel IO in the last years. But it still is not a 'plug-and-play' solution if you are looking for best performance. Without tuning it can easily slow down your IO compared to pure MPI-IO tremendously.
But the benefits of HDF5 over pure MPI-IO are huge: http://www.hdfgroup.org/why_hdf
(and in combination with XDMF even larger)
That´s why we wanted and had to stick with HDF5, even for our large computations with restart files > 2TByte each. Therefore the wrapper library parIO was developed to keep the IO of our applications simple, while getting best read/write performance on different supercomputers. It currently provides functionality to dump 1,2,3,4D arrays of different types, distributed over any number of MPI threads or not, as fast as possible to file while keeping the details of tuning away from the application developer.
It is used for IO in several large scale applications of the ITV on the supercomputers JUQUEEN at the Research Center Jülich and the SuperMUC at the Leibniz-Rechenzentrum and results in 15-30 GByte/s read/write performance for real life production runs.
parIO is licensed under the LGPL (GNU Lesser General Public License) and can be downloaded ->here<-
Please send me your feedback and let me know how parIO works for you.
Do not hesitate and contact me for any further questions, bug reports or patches.
Jens Henrik Göbbert, jh.goebbert(at)itv.rwth-aachen.de
features
- simplifies the use and tuning of HDF5
- takes care of HDF5s specific settings and functions to speedup IO
e.g buffer, padding, chunking, reduce write of metadata ... - takes care of HDF5s hyperslab support
distribution of single array over multiple processors - takes care of HDF5s different IO strategies
MPI-IO, POSIX, file-splitting(family), direct-IO - provides overloaded functions for different type and dimension
real4,real8,integer4,complex4,complex8 (0D,1D,2D,3D,4D) - simplifies setting of attributes
- provides much faster precision conversion of large arrays
read/write float from double
- takes care of HDF5s specific settings and functions to speedup IO
- detailed error handling
- detailed IO tests
- detailed IO informations
examples
e.g. reading two values from file to all MPI threads:
if(open_group(file_id, '/settings', group_id) ) then
call r_gval(group_id, 'nhalt', nhalt, ierr)
call r_gval(group_id, 'nstat', nstat, ierr)
call close_group(group_id, ierr)
end if
e.g. parallel read of three large arrays 'u','v','w', decomposed over all MPI threads (hyperslab) to memory:
if(open_group(file_id, '/flow', group_id)) then
call r_pval(group_id, 'u', foffs, mdims, mem, ierr);
call r_pval(group_id, 'v', foffs, mdims, mem, ierr)
call r_pval(group_id, 'w', foffs, mdims, mem, ierr)
call close_group(group_id, ierr)
end if
Here 'foffs' are the file offsets and 'mdims' are the dimensions of the array 'mem' to write to.
If only parts of the array 'mem' are supposed to be filled with data read from file 'msize' and 'moffs' has to be specified additionally:
if(open_group(file_id, '/flow', group_id)) then
call r_pval(group_id, 'u', foffs, mdims, mem, msize, moffs, ierr);
call r_pval(group_id, 'v', foffs, mdims, mem, msize, moffs, ierr)
call r_pval(group_id, 'w', foffs, mdims, mem, msize, moffs, ierr)
call close_group(group_id, ierr)
end if
Here 'msize' is the size of the data to read and 'moffs' are the memory offsets describing the position in memory to write to.
Download parIO and have at the example or check 'testio_io_basics.F90' to have an idea, how simple your IO routines will look like.
download
parIO 1.3.4 (04.04.2014)