https://github.com/CaNS-World/CaNS
A code for fast, massively-parallel direct numerical simulations (DNS) of canonical flows
https://github.com/CaNS-World/CaNS
cfd computational-fluid-dynamics fluid-dynamics fluid-simulation fortran gpu gpu-computing high-performance-computing turbulence
Last synced: 3 months ago
JSON representation
A code for fast, massively-parallel direct numerical simulations (DNS) of canonical flows
- Host: GitHub
- URL: https://github.com/CaNS-World/CaNS
- Owner: CaNS-World
- License: mit
- Created: 2017-06-26T20:40:51.000Z (almost 8 years ago)
- Default Branch: main
- Last Pushed: 2024-04-21T08:00:20.000Z (about 1 year ago)
- Last Synced: 2024-05-20T03:14:54.797Z (about 1 year ago)
- Topics: cfd, computational-fluid-dynamics, fluid-dynamics, fluid-simulation, fortran, gpu, gpu-computing, high-performance-computing, turbulence
- Language: Fortran
- Homepage:
- Size: 849 KB
- Stars: 181
- Watchers: 15
- Forks: 62
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-fluid-dynamics - p-costa/CaNS - A code for fast, massively-parallel direct numerical simulations (DNS) of canonical flows.  (Computational Fluid Dynamics / Finite Difference Methods (FDM))
README
## Synopsis
**CaNS (Canonical Navier-Stokes)** is a code for massively-parallel numerical simulations of fluid flows. It aims at solving any fluid flow of an incompressible, Newtonian fluid that can benefit from a FFT-based solver for the second-order finite-difference Poisson equation in a 3D Cartesian grid. In two directions the grid is regular and the solver supports the following combination of (homogeneous) boundary conditions:
* Neumann-Neumann
* Dirichlet-Dirichlet
* Neumann-Dirichlet
* Periodic
In the third domain direction, the solver is more flexible as it uses Gauss elimination. There the grid can also be non-uniform (e.g. fine at the boundary and coarser in the center).
CaNS also allows for choosing an implicit temporal discretization of the momentum diffusion terms, either fully implicit or only along the last domain direction. This results in solving a 3D/1D Helmholtz equation per velocity component. In the fully implicit case, FFT-based solvers are also used, and the same options described above for pressure boundary conditions apply to the velocity.
**Reference**
P. Costa. *A FFT-based finite-difference solver for massively-parallel direct numerical simulations of turbulent flows.* *Computers & Mathematics with Applications* 76: 1853--1862 (2018). [doi:10.1016/j.camwa.2018.07.034](https://doi.org/10.1016/j.camwa.2018.07.034) [[arXiv preprint]](https://arxiv.org/abs/1802.10323)
## News
**[29/01/2025]:** To avoid having the `build.conf` and `configs/*.mk` files - often changed by the user - tracked by git, the compiling behavior has changed slightly; see Compilation below.
**[28/01/2025]:** Input file `input.nml` has changed slightly, to allow for prescribing a fixed time step size `dt_f`.
### _Major Update:_ [`CaNS 2.0`](docs/CaNS-2.0.md) _is finally out!_ :tada:
**`CaNS 2.0` has many new features, being the result of the most significant revision effort undertaken so far.** It includes major improvements in performance and robustness, and a fresh hardware-adaptive many-GPU parallelization using the [*cuDecomp*](https://github.com/NVIDIA/cuDecomp) library. See [`docs/CaNS-2.0.md`](docs/CaNS-2.0.md) for a detailed description of all new features. CaNS 2.0 has been tested and observed to run efficiently on some major GPU-accelerated clusters such as Perlmutter, Summit, and Marconi 100.
## Features
Some features are:
* Hybrid MPI/OpenMP parallelization
* FFTW guru interface / cuFFT used for computing multi-dimensional vectors of 1D transforms
* The right type of transformation (Fourier, cosine, sine, etc) is automatically determined form the input file
* [cuDecomp](https://github.com/NVIDIA/cuDecomp) pencil decomposition library for _hardware-adaptive_ distributed memory calculations on _many GPUs_
* [2DECOMP&FFT](https://github.com/xcompact3d/2decomp-fft) library used for performing global data transpositions on CPUs and some of the data I/O
* GPU acceleration using OpenACC directives
* A different canonical flow can be simulated just by changing the input files
Some examples of flows that this code can solve are:
* periodic or developing channel
* periodic or developing square duct
* tri-periodic domain
* lid-driven cavity
## Motivation
This project aimed first at being a modern alternative to the well-known FISHPACK routines (Paul Swarztrauber & Roland Sweet, NCAR) for solving a three-dimensional Helmholtz equation. After noticing some works simulating canonical flows with iterative solvers -- when faster direct solvers could have been used instead -- it seemed natural to create a versatile tool and make it available. This code can be used as a first base code for which solvers for more complex flows can be developed (e.g. extensions with fictitious domain methods).
## Method
The fluid flow is solved with a second-order finite difference pressure correction scheme, discretized in a MAC grid arrangement. Time is advanced with a three-step low storage Runge-Kutta scheme. Optionally, for increased stability at low Reynolds numbers, at the price of higher computational demand, the diffusion term can be treated implicitly. See the reference above for details.
## Usage
### Downloading *CaNS*
Since *CaNS* loads the external pencil decomposition libraries as Git Submodules, the repository should be cloned as follows:
```bash
git clone --recursive https://github.com/CaNS-World/CaNS
```
so the libraries are downloaded too. Alternatively, in case the repository has already been cloned without the Submodules (i.e., folders `cuDecomp` and `2decomp-fft` under `dependencies/` are empty), the following command can be used to update them:
```bash
git submodule update --init --recursive
```
### Compilation
#### Prerequisites
The prerequisites for compiling CaNS are the following:
* MPI
* FFTW3/cuFFT library for CPU/GPU runs
* The `nvfortran` compiler (for GPU runs)
* CMake for compiling the cuDecomp library (for GPU runs)
* NCCL and NVSHMEM (optional, may be exploited by the cuDecomp library)
* OpenMP (optional)
#### In short
For most systems, CaNS can be compiled from the root directory with the following commands `make libs && make`, which will compile the 2DECOMP&FFT/cuDecomp libraries, and CaNS.
#### Detailed instructions
The `Makefile` in root directory is used to compile the code, and is expected to work out-of-the-box for most systems. The `build.conf` file in the root directory can be used to choose the Fortran compiler (MPI wrapper), and a few pre-defined profiles depending on the nature of the run (e.g., production vs debugging), and pre-processing options; see [`INFO_COMPILING.md`](docs/INFO_COMPILING.md) for more details. The default `build.conf` file is created from `configs/defaults/build-default.conf` at the first compilation. Concerning the pre-processing options, the following are available:
* `DEBUG` : performs some basic checks for debugging purposes
* `TIMING` : wall-clock time per time step is computed
* `IMPDIFF` : diffusion terms are integrated implicitly in time (thereby improving the stability of the numerical algorithm for viscous-dominated flows)
* `IMPDIFF_1D` : same as above, but with implicit diffusion *only* along Z; *for optimal parallel performance, the domain should not be decomposed along Z*
* `PENCIL_AXIS` : sets the default pencil direction, one of [1,2,3] for [X,Y,Z]-aligned pencils; X-aligned is the default and should be optimal for all cases except for Z implicit diffusion, where using Z-pencils is recommended if `dims(2) > 1` in the input file
* `SINGLE_PRECISION` : calculation will be carried out in single precision (the default precision is double)
* `GPU` : enable GPU-accelerated runs
### Input file
The input file `input.nml` sets the physical and computational parameters. In the `examples/` folder are examples of input files for several canonical flows. See [`INFO_INPUT.md`](docs/INFO_INPUT.md) for a detailed description of the input file.
Files `out1d.h90`, `out2d.h90` and `out3d.h90` in `src/` set which data are written in 1-, 2- and 3-dimensional output files, respectively. *The code should be recompiled after editing out?d.h90 files*.
### Running the code
Run the executable with `mpirun` with a number of tasks complying to what has been set in the input file `dns.in`. Data will be written by default in a folder named `data/`, which must be located where the executable is run (by default in the `run/` folder).
### Visualizing field data
See [`INFO_VISU.md`](docs/INFO_VISU.md).
## Contributing
We appreciate any contributions and feedback that can improve CaNS. If you wish to contribute to the tool, please get in touch with the maintainers or open an Issue in the repository / a thread in Discussions. Pull Requests are welcome, but please propose/discuss the changes in an linked Issue first.
## Final notes
Please read the `ACKNOWLEDGEMENTS`, `LICENSE` files.