Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/wkliao/parallel-netcdf

Mirror to PnetCDF SVN repo. PnetCDF official repo is now in https://github.com/Parallel-NetCDF/PnetCDF
https://github.com/wkliao/parallel-netcdf

fortran io mpi netcdf parallel-computing

Last synced: about 2 months ago
JSON representation

Mirror to PnetCDF SVN repo. PnetCDF official repo is now in https://github.com/Parallel-NetCDF/PnetCDF

Awesome Lists containing this project

README 

        

Parallel netCDF (PnetCDF) is a library providing high-performance parallel I/O

while still maintaining file-format compatibility with Unidata's NetCDF,

specifically the classic CDF 1, 2, and 5 formats. Although NetCDF supports

parallel I/O starting from version 4, it is built on top of HDF5 and thus its

parallel feature requires files in HDF5 format. PnetCDF is currently the only

option for parallel I/O on files in classic formats.



Starting from version 1.1.0, PnetCDF supports the CDF-5 file format, an

extension of CDF-2, that supports more atomic data types and allows users to

define large dimensions, attributes, and variables (>2B array elements).



NetCDF gives scientific programmers a space-efficient and portable means for

storing data.  By making some small changes to the NetCDF APIs, PnetCDF can use

MPI-IO to achieve high-performance parallel I/O.



More extensive installation guides can be found in file INSTALL.  Additional

information regarding the contents of the release can be found in the

RELEASE_NOTES file in the top-level directory.  Finally, the PnetCDF project

web sites below contain information on bug fixes and software releases.

    http://trac.mcs.anl.gov/projects/parallel-netcdf

    http://cucis.ece.northwestern.edu/projects/PnetCDF



Requirements:

    - PnetCDF relies on MPI-IO.  ROMIO, the most common MPI-IO implementation,

      can make use of a prefix in front of the file name to specify the

      underlying file system and override ROMIO's auto detection. A typical

      prefix would look like "nfs:" or "pvfs2:".  Bear this in mind if you use

      a file name with ':' characters.  In some cases, ROMIO might think you

      are passing in a prefix and will complain about an unsupported file

      system type.  If that happens, add a file-system prefix to your file

      name.

    - To build PnetCDF, you will need the following additional programs:

        - autotools (autoconfig, automake, libtool, m4, etc.)

        - Optional: (for PnetCDF developers)

             - either yacc or bison

	     - either lex or flex

      These are usually part of your operating system's development tools.



Build recipes:

    The process of building and installation of PnetCDF library is usually

    simple: by just running commands "./configure", "make", and "make install".

    (If more than one set of MPI compilers are available on your system, adding

    option --with-mpi=/path/to/MPI/install to the configure command line is

    recommended.) Several configure options are also available to customize it.

    See INSTALL for more information.



    Several machine- and compiler-specific build recipes are available under

    the directory of doc.

    - README.IBM        (BlueGene systems)

    - README.SX         (NEC SX systems)

    - README.CRAY       (Cray systems)

    - README.INTEL      (Intel Compilers)

    - README.LINUX      (GNU compilers)

    - README.Fujitsu    (Fujitsu systems)

    - README.K-Computer (the K computer @RIKEN)

    - README.SGI        (SGI systems, such as Endeavour @NASA)

    - README.Ubuntu     (Ubuntu)



PnetCDF User Documents:

    C API References

      http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c

    Questions & Answers

      http://cucis.ece.northwestern.edu/projects/PnetCDF/faq.html



Note on supporting large files and large variables.

    - Some Fortran APIs contain arguments of 64-bit integer data type, which

      corresponds to MPI_Offset data type in their peer C APIs. Declare those

      arguments as 'integer(kind=MPI_OFFSET_KIND)'.



    - In places where you might use NF_UNLIMITED to define an unlimited

      dimension in one of the nfmpi_ routines, use NFMPI_UNLIMITED.

      NFMPI_UNLIMITED has been defined as the proper 64-bit integer type for

      nfmpi_def_dim, whereas NF_UNLIMITED might be too small. Similarly, for

      Fortran 90, use NF90MPI_UNLIMITED instead of NF90_UNLIMITED.



Mailing List

    PnetCDF user community discusses the design and use of the library on the

    [email protected] mailing list. Anyone interested in developing

    or using PnetCDF is encouraged to join. Visit the list information page

    https://lists.mcs.anl.gov/mailman/listinfo/parallel-netcdf

    for subscription instructions. This mailing list is also for announcements,

    bug reports, and questions about PnetCDF software.



    The past discussions in the mailing list are available in the follow URL:

    http://lists.mcs.anl.gov/pipermail/parallel-netcdf/.