Hybrid MPI+OpenMP parallelization of a Navier-Stokes solver for large-scale DNS

Abstract

This work presents a parallel Navier-Stokes solver for the large-scale direct numerical simulation (DNS) of incompressible flows with one periodic direction. It is based on a scalable Poisson solver that combines an FFT diagonalization with a preconditioned conjugate gradient (PCG) method and a direct Schurcomplement based method. Modern architectures of supercomputers with multi-core nodes motivate the use of a two-level hybrid MPI+OpenMP parallelization. The use of the more complex two-level approach has allowed to significantly extend the number of CPUs the solver can efficiently use. Here, the solver has been successfully tested on up to 12800 CPU cores for meshes with up to 109 grid points. However, estimations based on presented results show that more than 50000 CPU cores can potentially be exploited.