Section: New Software and Platforms

AeroSol

Participants : Simon Delmas [Cagire] , Benjamin Lux [Cagire] , Nikolaos Pattakos [Cardamom] , Vincent Perrier [Cagire, correspondent] , Mario Ricchiuto [Cardamom] .

Developed since 2011 by V. Perrier in partnership with the Cardamom Inria team, the AeroSol library is a high order finite element library written in C++. The code design has been carried for being able to perform efficient computations, with continuous and discontinuous finite element methods on hybrid and possibly curvilinear meshes.

The work of the Cardamom team is focused on continuous finite element methods, while we focus on discontinuous Galerkin methods. However, everything is done for sharing the largest possible part of code. The distribution of the unknowns is made with the software PaMPA, first developed within the Inria teams Bacchus and Castor, and currently maintained in the Tadaam team.

The generic features of the library are

• High order. It can be theoretically any order of accuracy, but the finite element basis, and quadrature formula are implemented for having up to a fifth order of accuracy.

• Hybrid and curvilinear meshes. AeroSol can deal with up to fifth order conformal meshes composed of lines, triangles, quadrangles, tetrahedra, hexahedra, prism, and pyramids.

• Continuous and discontinuous discretization. AeroSol deals with both continuous and discontinuous finite element methods.

We would like to emphasize three assets of this library:

• Its development environment For allowing a good collaborative work and a functional library, a strong emphasis has been put on the use of modern collaborative tools for developing our software. This includes the active use of a repository, the use of CMake for the compilation, the constant development of unitary and functional tests for all the parts of the library (using CTest), and the use of the continuous integration tool Jenkins for testing the different configurations of AeroSol and its dependencies. Efficiency is regularly tested with direct interfacing with the PAPI library or with tools like scalasca.

• Its genericity A lot of classes are common to all the discretization, for example classes concerning I/O, finite element functions, quadrature, geometry, time integration, linear solver, models and interface with PaMPA. Adding simple features (e.g. models, numerical flux, finite element basis or quadrature formula) can be easily done by writing the class, and declaring its use in only one class of the code.

• Its efficiency This modularity is achieved by means of template abstraction for keeping good performances. Dedicated efficient implementation, based on the data locality of the discontinuous Galerkin method has been developed. As far as parallelism is concerned, we use point-to-point communications, the HDF5 library for parallel I/O. The behavior of the AeroSol library at medium scale (1000 to 2000 cores) was studied in [19] .

The AeroSol project fits with the first axis of the Bordeaux Sud Ouest development strategy, which is to build a coherent software suite scalable and efficient on new architectures, as the AeroSol library relies on several tools developed in other Inria teams, especially for the management of the parallel aspects.

At the end of 2014, AeroSol had the following features:

• Development environment Use of CMake for compilation (gcc, icc and xlc), CTest for automatic tests and memory checking, lcov and gcov for code coverage reports. Development of a CDash server for collecting the unitary tests and the memory checking. Beginning of the development of an interface for functional tests. Optional linking with HDF5, PAPI, with dense small matrices libraries (BLAS, Eigen)

• In/Out Link with the XML library for handling with parameter files. Parallel reader for GMSH, with an embedded geometrical pre-partitioner. Writer on the VTK-ASCII legacy format (cell and point centered). Parallel output in vtu and pvtu (Paraview) for cell-centered visualization, and XDMF/HDF5 format for both cell and point centered visualization. Ability of saving the high order solution and restarting from it. Computation of volumic and probe statistics. Ability of saving averaged layer data in quad and hexa meshes. Ability of defining user defined output visualization variables.

• Quadrature formula up to 11th order for Lines, Quadrangles, Hexaedra, Pyramids, Prisms, up to 14th order for tetrahedron, up to 21st order for triangles. Gauss-Lobatto type quadrature formula for lines, triangles, quadrangles and hexaedra.

• Finite elements up to fourth degree for Lagrange finite elements and hierarchical orthogonal finite element basis (with Dubiner transform on simplices) on lines, triangles, quadrangles, tetrahedra, prisms, hexaedra and pyramids. Finite element basis that are interpolation basis on Gauss-Legendre points for lines, quadrangles, and hexaedra, and triangle (only 1st and 2nd order).

• Geometry Elementary geometrical functions for first order lines, triangles, quadrangles, prisms, tetrahedra, hexaedra and pyramids. Handling of high order meshes.

• Time iteration explicit Runge-Kutta up to fourth order, explicit Strong Stability Preserving schemes up to third order. Optimized CFL time schemes: SSP(2,3) and SSP(3,4). CFL time stepping. Implicit integration with BDF schemes from 2nd to 6th order Newton method for stationary problems. Implicit unstationary time iterator non consistent in time for stationary problems. Implementation of in house GMRES and conjugate gradient based on Jacobian free iterations.

• Linear Solvers Link with the external linear solver UMFPack, PETSc and MUMPS. Internal solver for diagonal and block-diagonal matrices.

• Memory handling discontinuous and continuous, sequential and parallel discretizations based on PaMPA for generic meshes, including hybrid meshes.

• Models Perfect gas Euler system, real gas Euler system (template based abstraction for a generic equation of state), scalar advection, Waves equation in first order formulation, generic interface for defining space-time models from space models. Diffusive models: isotropic and anisotropic diffusion, compressible Navier-Stokes. Scalar advection-diffusion model.

• Numerical schemes Continuous Galerkin method for the Laplace problem (up to fifth order) with non consistent time iteration or with direct matrix inversion. Explicit and implicit discontinuous Galerkin methods for hyperbolic systems, diffusive and advection-diffusion problems. Beginning of optimization by stocking the geometry for advection problems. SUPG and Residual disribution schemes. Optimization of DG schemes for advection-diffusion problems: stocking of the geometry and use of BLAS for all the linear phases of the scheme.

• Numerical fluxes Centered fluxes, exact Godunov' flux for linear hyperbolic systems, and Lax-Friedrich flux. Riemann solvers for Low Mach flows. Numerical flux accurate for steady and unsteady computations.

• Boundary conditions Periodic boundary conditions, time-dependent inlet and outlet boundary conditions. Adiabatic wall and isothermal wall. Steger-Warming based boundary condition.

• Parallel computing Mesh redistribution, computation of Overlap with PaMPA. Collective asynchronous communications (PaMPA based). Asynchronous point to point communications. Tests on the cluster Avakas from MCIA, and on Mésocentre de Marseille, and PlaFRIM. Tier-1 Turing (BlueGene).

• C++/Fortran interface Tests for binding fortran with C++.

• Instrumentation Aerosol can give some traces on memory consumption/problems with an interfacing with the PAPI library. Tests have also been performed with VTUNE and TAU. Tests with Maqao and Scalasca (VIHPS workshop).

• Validation Poiseuille, Taylor-Green vortex. Laplace equation on a ring and Poiseuille flow on a ring. Implementation of volumic forcing based on wall dissipation.

In 2015, N. Pattakos was hired in the team Cardamom, in order to improve the code architecture and for easing the installation of the library. The following features were added or improved:

• Development environment The use of CMake was strongly improved, which induced also easier test launching. Documentation, code cleaning and refactorization have also been led. The shared project of Plafrim was updated, and so was the joint Aerosol/Scotch/PaMPA project on the continuous integration platform. Integration of SPack for handling dependencies has begun. Interface with ESSL was fixed.

• Multigrid Development of $p$-multigrid methods. This includes also the possibility of beginning a computation iwth an order and to decrease or increase the order of approximation when restarting. For the $p$ multigrid methods, $V$ and $W$ cycle have been developed, and restriction and prolongation opertors have also been developed. Implementation of $h$-multigrid has started, with the development of tests of the aggregation methods of PaMPA, and the definition of finite element basis on arbitrary cells.

• Boundary conditions Development of the Synthetic Eddy Method boundary condition.

• Models Linearized Euler equations, and Sutherland model for non isothermal diffusive flows. Shallow-water model.

• Parallel computing Weighted load balancing for hybrid meshes.

• Validation Turbulent channel flow.

• Postprocessing Development of high order projections over line postprocessing, possibility of stocking averaged data, such as the average flow and the Reynolds stresses.