Strategies for the heterogeneous execution of large-scale simulations on hybrid supercomputers

Abstract

Massively-parallel devices of various architectures are being adopted by the newest supercomputers to overcome the actual power constraint in the context of the exascale challenge. This progress leads to an increasing hybridisation of HPC systems and makes the design of computing applications a rather complex problem. Therefore, the software e¿ciency and portability are of crucial importance. In this context of accelerated innovation, we developed the HPC2 (Heterogeneous Portable Code for HPC). It is a portable, algebra-based framework for heterogeneous computing with many potential applications in the ¿elds of computational physics and mathematics, such as modelling of incompressible turbulent ¿ows. In its application to CFD, the algorithm of the timeintegration phase relies on a reduced set of only three algebraic operations: the sparse matrix-vector product, the linear combination of vectors and the dot product. This algebraic approach combined with a multilevel MPI+OpenMP+OpenCL parallelisation naturally provides modularity and portability. In this work, we present the strategies for the e¿cient heterogeneous execution of large-scale simulations on hybrid supercomputers that are part of the HPC2 core. The performance is studied in detail for the major computing kernel, the sparse matrix-vector product, using a sparse matrix derived from a simulation on a hybrid unstructured mesh and up to 32 nodes of a hybrid CPU+GPU supercomputer.