2025​‌Activity reportProject-TeamTOPAL​​

3.1‌​‌ Objectives

We propose to​​ structure our research around​​​‌ two main application fields‌ (see Section 4):‌​‌ linear multi-dimensional algebra and​​ solvers on the one​​​‌ hand, and training in‌ particular of deep learning‌​‌ networks on the other​​ hand. In these two​​​‌ domains, our contributions will‌ be organized around three‌​‌ main research axes (see​​ Section 3.3): the​​​‌ use of task based‌ runtime systems (to provide‌​‌ robust solutions and to​​ increase the portability in​​​‌ the context of heterogeneous‌ large scale platforms), the‌​‌ use of compression (to​​ limit memory footprint and​​​‌ data transfers) and the‌ minimization of energy consumption‌​‌ and carbon impact (using​​ an approach of rewriting​​​‌ algorithms and placement strategies‌ to limit data movements).‌​‌ This matrix organization of​​​‌ our activities (see Section​ 3.4) is intended​‌ to maximize the interactions​​ between the different researchers​​​‌ of the team and​ facilitate knowledge sharing and​‌ joint participation in projects.​​

In these topics, the​​​‌ use of task based​ runtime systems and the​‌ design of efficient linear​​ algebra kernels and solvers​​​‌ belong to the historical​ expertise of the team​‌ and is shared by​​ all team members, especially​​​‌ in the context of​ linear algebra kernels. Our​‌ goal is to build​​ on this expertise to​​​‌ extend the use of​ task based runtime systems​‌ to other types of​​ applications such as training​​​‌ and to use the​ precise knowledge of these​‌ linear algebra kernels to​​ incorporate new criteria such​​​‌ as energy minimization. The​ application to training (and​‌ interference) in deep neural​​ networks and data compression​​​‌ are subjects we have​ been interested in for​‌ a few years, typically​​ during the last HiePACS​​​‌ evaluation period and within​ the Inria Challenge of​‌ AI, HPC and Big​​ Data led by Bruno​​​‌ Raffin. The extension of​ the techniques developed in​‌ linear algebra to tensor​​ algebra and tensor decompositions​​​‌ is natural, given the​ proximity of the fields​‌ and the practical importance​​ of the subject, but​​​‌ it is more recent​ and reinforced by the​‌ arrival of Julia Gusak,​​ who is an expert​​​‌ in the field. Finally,​ the objective of energy​‌ and carbon footprint minimization,​​ at the algorithmic and​​​‌ software levels rather than​ at the architecture level,​‌ is a field that​​ we wish to emphasize​​​‌ in our research, both​ because of its own​‌ fundamental importance and because​​ we believe that our​​​‌ expertise and the techniques​ that we have developed​‌ in recent years are​​ well adapted to it​​​‌ and that the approach​ we propose is original.​‌

3.2 Overall Positionning

The​​ general positioning of the​​​‌ team is to produce​ tools for users, academic​‌ or industrial, in the​​ form of algorithms and​​​‌ software libraries. These users​ can work either in​‌ numerical simulation or in​​ training. Nevertheless, as our​​​‌ experiences in simulation and​ training have already demonstrated,​‌ this interaction cannot be​​ carried out in the​​​‌ form of providing black​ boxes and it is​‌ crucial for us to​​ work directly with the​​​‌ users of our software​ to understand their needs​‌ and adapt our algorithms​​ and codes to the​​​‌ characteristics of their data.​ This interaction will be​‌ particularly critical to work​​ on data representation and​​​‌ compression, which requires a​ strong interaction with numerical​‌ methods and machine learning​​ in order to understand​​​‌ the application requirements and​ the characteristics of data,​‌ based on their significance.​​

At the other end​​​‌ of the spectrum, it​ is also essential for​‌ us to maintain close​​ relationships with both the​​​‌ architecture and system communities.​ Indeed, the very rapid​‌ growth of machine learning​​ applications has also renewed​​​‌ the landscape of computing​ resources with the emergence​‌ of very original solutions,​​ at the architectural and​​​‌ arithmetic level. Even if​ we cannot influence on​‌ these evolutions, it is​​ very important to propose​​ solutions that make the​​​‌ best use of them.‌ We also decided several‌​‌ years ago to rely​​ on task based runtime​​​‌ systems to implement our‌ software developments. This decision‌​‌ has many implications on​​ our developments and requires​​​‌ an extremely close collaboration‌ with their designers. In‌​‌ this context, we have​​ co-supervised several PhD theses​​​‌ related to StarPU with‌ the Storm project team‌​‌ and we will pursue​​ this strategy, which is​​​‌ crucial in particular to‌ take into account the‌​‌ challenges ahead of us:​​ the transition to exascale,​​​‌ the integration of the‌ energy, the extension to‌​‌ training applications and the​​ ever increasing heterogeneity of​​​‌ computing resources.

3.3 Research‌ Axes

3.4 Main‌ Research Topics

The list‌​‌ of our contributions can​​ be read at the​​​‌ intersection of the research‌ domains described in Section‌​‌ 4 and research axes​​ described in Section 3.3​​​‌ as shown in the‌ following table:

4.1 Multi-Linear Algebra‌ and Solvers

Participants: Olivier‌​‌ Beaumont, Aurélien Esnard​​, Lionel Eyraud Dubois​​​‌, Mathieu Faverge,‌ Abdou Guermouche, Yulia‌​‌ Gusak, Thomas Herault​​, Pierre Ramet,​​​‌ Philippe Swartvagher.

At‌ the core of a‌​‌ large number of simulation​​ tools, the resolution of​​​‌ large linear systems often‌ represents the dominant part‌​‌ of the computing time.​​ These linear solvers rely​​​‌ on a wide variety‌ of numerical methods and‌​‌ algorithms. Massively parallel versions​​ are required to support​​​‌ advances in multi-physics and‌ multi-scale simulations, especially when‌​‌ targeting exascale platforms. The​​ aim is therefore to​​​‌ address the major challenge‌ of designing and building‌​‌ numerically robust solvers on​​ top of runtime systems​​​‌ that can scale up‌ and push back the‌​‌ limits of existing industrial​​ codes by making full​​​‌ use of all computing‌ resources such as CPUs,‌​‌ GPUs and other accelerator​​ units. Following the ANR​​​‌ project SOLHARIS (and previously‌ SOLHAR), we now have‌​‌ experience of strong/weak scalability​​ of sparse direct solvers​​​‌ on large scale, distributed‌ memory, heterogeneous computers. These‌​‌ solvers already rely on​​ asynchronous task-based parallelism 48​​​‌, 49, 78‌, 47, rather‌​‌ than traditional and widely​​ adopted message-passing and multithreading​​​‌ techniques. Indeed, the use‌ of modern runtime systems‌​‌ have proven to be​​ good tools for the​​​‌ development of scientific computing‌ applications 80, 62‌​‌, 86, in​​ particular in combination with​​​‌ compression 63, 85‌, 84, 59‌​‌, 76 and communication​​ avoiding techniques 60,​​​‌ 53. This work‌ can be extended naturally‌​‌ to multi-dimensional objects such​​ as tensors. In the​​​‌ tensor case, we propose‌ to extend the data‌​‌ distribution strategies to minimize​​ communication and the use​​​‌ of system runtimes to‌ handle the variability and‌​‌ heterogeneity of computational resources.​​ Finally, we have focused​​​‌ so far on minimizing‌ the execution time, whereas‌​‌ energy efficiency is becoming​​ a critical element. We​​​‌ therefore plan to revisit‌ the algorithms and methods‌​‌ we developed in linear​​ algebra, and those we​​​‌ propose to design for‌ handling tensors, to allow‌​‌ the optimal use of​​ the available hardware in​​​‌ order to guarantee the‌ performance of the computations‌​‌ within a fixed energy​​ budget.

4.2 Training and​​​‌ Inference for DNNs

Participants:‌ Olivier Beaumont, Lionel‌​‌ Eyraud Dubois, Yulia​​ Gusak, Thomas Herault​​​‌, Laércio Lima Pilla‌, Pierre Ramet,‌​‌ Philippe Swartvagher.

The​​ training phase in Deep​​​‌ Neural Networks has become‌ an important source of‌​‌ HPC resource usage and​​ it is crucial to​​​‌ perform it efficiently on‌ parallel architectures. Until today,‌​‌ data parallelism is the​​ most widely used method,​​​‌ but the associated requirement‌ to replicate all the‌​‌ weights on all computing​​ resources causes memory issues​​​‌ at the level of‌ each node and of‌​‌ collective communications at the​​ level of the platform.​​​‌

In general, the overall‌ shape of the dependency‌​‌ graphs associated with the​​ feed forward training phase​​​‌ has characteristics (long dependencies)‌ that generate a lot‌​‌ of memory needs and​​​‌ data exchange. However, there​ are multiple opportunities to​‌ address these problems by​​ combining 55 re-computations 66​​​‌, 54, 61​, 77, 72​‌, 5, offloading​​ 57, compression and​​​‌ different parallelism strategies (image,​ filter, kernel, model parallelism​‌ 58, 81,​​ 56, 74,​​​‌ 36). It is​ also promising to consider​‌ other more radical techniques​​ to go beyond feed​​​‌ forward training, such as​ the use of multigrid​‌ reduction in time (MGRIT)​​ 68, 69 that​​​‌ come from the field​ of numerical simulations and​‌ that we already address​​ in other contexts.

Within​​​‌ this general framework, the​ minimization of carbon footprint​‌ is obviously a major​​ concern that must guide​​​‌ strategies. Tools to train​ complex and deep network​‌ on otherwise obsolete hardware​​ using memory saving techniques​​​‌ are already a strong​ contribution in this direction​‌ to increase the lifetime​​ of computing resources. and​​​‌ our goal is to​ extend these techniques in​‌ terms of efficiency and​​ in terms of scope,​​​‌ which has consumed a​ little more energy associated​‌ with the computations. As​​ in the case of​​​‌ linear algebra, energy optimization​ also requires the use​‌ of heterogeneous computation resources​​ (CPUs, GPUs, TPUs, FPGAs).​​​‌ Conversely, this heterogeneity hinders​ scalability because of difficulties​‌ in predicting task durations​​ and makes the use​​​‌ of dynamic runtime schedulers​ necessary. Finally, the use​‌ of these dynamic runtimes​​ also poses the problem​​​‌ of knowing what needs​ to be decided statically​‌ and dynamically in terms​​ of resource allocation and​​​‌ scheduling.

5.1 Footprint​‌ of research activities

As​​ part of our research​​​‌ activities, we use local​ computing resources such as​‌ PlaFRIM and the national​​ computing resources of IDRIS​​​‌ and the TGCC.​

The environmental impact of​‌ using these platforms is​​ significant, whether for numerical​​​‌ simulation or training applications.​ However, the positioning of​‌ the team, which produces​​ simulation and training tools​​​‌ but does not directly​ perform simulations and training,​‌ is relatively limited. For​​ example, in the case​​​‌ of training, we have​ so far concentrated on​‌ techniques that do not​​ modify the architecture of​​​‌ the networks and the​ computations that are performed,​‌ so that the number​​ of epochs and the​​​‌ final accuracy are not​ impacted. In this way,​‌ it is possible to​​ validate our developments to​​​‌ accelerate training on a​ single batch (at full​‌ machine scale) and then​​ to extrapolate the acceleration​​​‌ at the whole training​ scale. Similarly, the techniques​‌ developed in linear algebra​​ in the team often​​​‌ do not depend (typically​ for dense approaches) on​‌ the numerical properties of​​ the matrices, so that​​​‌ acceleration (for a given​ problem size) can be​‌ validated without heavy experimental​​ campaigns, beyond what is​​​‌ necessary to obtain valid​ experimental results in complex​‌ environments where performance varies​​ from one experiment to​​​‌ another.

In this context,​ the use of simulation​‌ as opposed to direct​​ experimentation is also a​​​‌ tool that enables us​ to limit the impact​‌ of our research on​​ power consumption, since simulation​​ can save several orders​​​‌ of magnitude in power‌ consumption compared with direct‌​‌ experimentation. In this context,​​ it is crucial to​​​‌ produce simulation tools that‌ are as precise and‌​‌ generic as possible, and​​ the team has been​​​‌ actively collaborating for many‌ years in the development‌​‌ of simulation tools such​​ as SimGRID.

Nevertheless,​​​‌ the tools we produce‌ are used on a‌​‌ large scale in terms​​ of computation resources and​​​‌ simulation/training time, and the‌ associated energy consumption issue‌​‌ is therefore indirectly crucial.​​ In this context, we​​​‌ are developing original solutions‌ for reusing the heat‌​‌ dissipated by computation resources,​​ in particular as part​​​‌ of the Inria-Qarnot Computing‌ Pulse challenge (see Section‌​‌ 5.2). We have​​ also added a research​​​‌ axis aimed at minimizing‌ energy consumption for a‌​‌ given kernel (Section 3.3.3​​).

TOPAL has also​​​‌ signed the "Labos en‌ transitions" Charter of Commitment‌​‌ for research facilities on​​ the Bordeaux university site​​​‌ whose preamble states that‌ "Faced with contemporary environmental‌​‌ and societal challenges, and​​ the urgent need for​​​‌ systemic transformation to meet‌ them, the academic world‌​‌ has a particular responsibility:​​ to promote responsible research,​​​‌ aware of environmental issues‌ and respectful of the‌​‌ people who produce it,​​ which contributes to transitions​​​‌ and enables us to‌ understand and guide current‌​‌ and future societal transformations".​​ In exchange for this​​​‌ commitment, the establishments undertake‌ to provide us with‌​‌ an estimate of the​​ impact of our research​​​‌ activities (including the purchase‌ of equipment and missions).‌​‌ At this stage, this​​ information is difficult to​​​‌ aggregate at team level,‌ but making it available‌​‌ will enable us to​​ measure our progress and​​​‌ involvement.

5.2 Impact of‌ research results

7.1 Latest software​​​‌ developments

8.1 Scalable and​‌ portable LU factorization with​​ partial pivoting on top​​​‌ of runtime systems (Topic​ 3.4.1)

Participants: Alycia​‌ Lisito, Mathieu Faverge​​, Pierre Ramet.​​​‌

Task-based runtime systems have​ demonstrated efficiency in leveraging​‌ the capabilities of large,​​ heterogeneous architectures. Many linear​​​‌ algebra algorithms and applications​ have been implemented on​‌ top of runtime systems​​ to increase their performance.​​​‌ However, the High Performance​ Linpack (HPL) benchmark, used​‌ by the TOP500 to​​ rank supercomputers, has not​​​‌ yet been successfully implemented​ using taskbased runtime systems.​‌ In this paper, we​​ explore solutions to implement​​​‌ efficient LU factorization with​ partial pivoting using the​‌ sequential task-flow programming model.​​ We show that, due​​​‌ to the pivoting strategy,​ this algorithm generates a​‌ large number of very​​ small tasks, which usually​​​‌ overload the runtime system​ and make it inefficient.​‌ We propose two solutions​​ to improve the efficiency​​​‌ and reduce the number​ of tasks. First, we​‌ apply wellknown blocking strategies​​ in the context of​​ task-based algorithms. Secondly, we​​​‌ explore batching techniques to‌ reduce the number of‌​‌ tasks submitted to the​​ runtime system. Moreover, in​​​‌ distributed architectures, partial pivoting‌ generates many reductions on‌​‌ the critical path throughout​​ the factorization which needs​​​‌ to be carefully handled‌ to reach high performance.‌​‌ Two task-based reduction algorithms​​ are proposed to express​​​‌ these operations and improve‌ the runtime reactivity on‌​‌ the critical path. These​​ proposals have been implemented​​​‌ in the dense linear‌ algebra library CHAMELEON on‌​‌ top of the STARPU​​ runtime system. Experiments conducted​​​‌ on our cluster with‌ these optimizations show that‌​‌ our LU with partial​​ pivoting asymptotically reaches the​​​‌ performance of the non-pivoting‌ algorithm.

This work has‌​‌ been presented at IPDPS​​ Conference, June 2025, Milan,​​​‌ Italy 20.

8.2‌ Batching the tasks of‌​‌ the LU factorization with​​ partial pivoting on top​​​‌ of runtime systems (Topic‌ 3.4.1)

Participants: Alycia‌​‌ Lisito, Mathieu Faverge​​, Florent Pruvost,​​​‌ Pierre Ramet.

Task-based‌ runtime systems have demonstrated‌​‌ efficiency in leveraging the​​ capabilities of large heterogeneous​​​‌ architectures. Many linear algebra‌ algorithms and applications have‌​‌ been implemented on top​​ of runtime systems to​​​‌ increase their performance. However,‌ the LU factorization with‌​‌ partial pivoting has not​​ yet been successfully implemented​​​‌ using task-based runtime systems.‌ This operation is used‌​‌ to solve large dense​​ linear systems in numerical​​​‌ simulations, such as the‌ Maxwell equations in electromagnetism.‌​‌ This factorization is a​​ major part of the​​​‌ High Performance Linpack (HPL)‌ benchmark used in the‌​‌ TOP500 to evaluate and​​ rank supercomputers. We explore​​​‌ solutions to implement efficient‌ LU factorization with partial‌​‌ pivoting using the sequential​​ task-flow programming model. These​​​‌ solutions have been implemented‌ in the dense linear‌​‌ algebra library Chameleon on​​ top of the StarPU​​​‌ runtime system. We showed‌ that, due to the‌​‌ pivoting strategy, this algorithm​​ generates a large number​​​‌ of very small tasks,‌ which usually overloads the‌​‌ runtime system and makes​​ it inefficient. With a​​​‌ naive task batching strategy,‌ we improved the efficiency‌​‌ and reduced the number​​ of tasks. We propose​​​‌ solutions to adapt the‌ batch size to the‌​‌ granularity of the tasks.​​ In order to do​​​‌ that, we first distinguish‌ two types of tasks‌​‌ and set an adapted​​ batch size for each.​​​‌ Then, we introduce a‌ heuristic based on the‌​‌ number of operations per​​ tasks to adapt the​​​‌ batch size to the‌ computational complexity of the‌​‌ tasks during the factorization.​​ Experiments conducted on our​​​‌ cluster with these optimizations‌ show that our LU‌​‌ factorization with partial pivoting​​ asymptotically reaches about 96%​​​‌ of the performance of‌ the non-pivoting algorithm. Thanks‌​‌ to the adaptive batch​​ size mechanism, the performance​​​‌ peak is reached even‌ faster.

This work has‌​‌ been presented at COMPAS​​ Conference, June 2025, Bordeaux,​​​‌ France 27.

8.3‌ Toward an algebraic multigrid‌​‌ method for the indefinite​​ Helmholtz equation (Topic 3.4.2​​​‌)

Participants: Clement Richefort‌, Pierre Ramet.‌​‌

It is well known​​ that multigrid methods are​​​‌ very competitive in solving‌ a wide range of‌​‌ SPD problems. However achieving​​​‌ such performance for non-SPD​ matrices remains an open​‌ problem. In particular, three​​ main issues may arise​​​‌ when solving a Helmholtz​ problem : some eigenvalues​‌ may be negative or​​ even complex, requiring the​​​‌ choice of an adapted​ smoother for capturing them,​‌ and because the near-kernel​​ space is oscillatory, the​​​‌ geometric smoothness assumption cannot​ be used to build​‌ efficient interpolation rules. Moreover,​​ the coarse correction is​​​‌ not equivalent to a​ projection method since the​‌ indefinite matrix does not​​ define a norm. We​​​‌ present some investigations about​ designing a method that​‌ converges in a constant​​ number of iterations with​​​‌ respect to the wavenumber.​ The method builds on​‌ an ideal reduction-based framework​​ and related theory for​​​‌ SPD matrices to improve​ an initial least squares​‌ minimization coarse selection operator​​ formed from a set​​​‌ of smoothed random vectors.​ A new coarse correction​‌ is proposed to minimize​​ the residual in an​​​‌ appropriate norm for indefinite​ problems. We also present​‌ numerical results at the​​ end of the paper.​​​‌

This paper has been​ published in SIAM SISC​‌ 11.

8.4 Hierarchical​​ partitioning for the numerical​​​‌ simulation of complex 3D​ objects (Topic 3.4.2)​‌

Participants: Dimitri Walther,​​ Mathieu Faverge, Pierre​​​‌ Ramet.

The Boundary​ Element Method (BEM) offers​‌ numerous advantages for simulating​​ complex physical phenomena. By​​​‌ placing the unknowns (or​ degrees of freedom) on​‌ the interfaces between different​​ media, it becomes possible​​​‌ to model problems with​ distant boundary conditions (such​‌ as fluid flow around​​ an object, acoustic or​​​‌ electromagnetic wave diffraction, radiative​ heat transfer, etc.). However,​‌ this approach results in​​ a fully coupled system​​​‌ with a dense matrix.​ When this dense matrix​‌ can be decomposed into​​ low-rank sub-blocks, it is​​​‌ possible to construct a​ hierarchical matrix (H-matrix) that​‌ approximates the original system​​ to a desired level​​​‌ of accuracy. In favorable​ scenarios, this approximation reduces​‌ spatial complexity from $\mathrm{O}(n^2)‌$ to $O(\mathrm{n}logn)$ by​‌ compressing the matrix sub-blocks.​​ This work investigates the​​​‌ relationship between the partitioning​ of degrees of freedom​‌ and the compression rate​​ of the H-matrix. A​​​‌ new hierarchical partitioning technique,​ specifically designed to optimize​‌ H-matrix compression, is introduced.​​ Unlike existing algorithms based​​​‌ on geometric information (such​ as Median cut, Cobblestone,​‌ or Space-filling curves), this​​ new method relies on​​​‌ the construction of a​ connectivity graph of the​‌ degrees of freedom. This​​ graph is built in​​​‌ quasi-linear time ($\mathrm{O}(nlog\mathrm{n‌})$) from the​​ mesh of the studied​​​‌ object and partitioned in​ log-quadratic time ($\mathrm{O‌}(n^{2}logn)$) using​​​‌ a multi-level partitioning approach.​ An additional constraint is​‌ imposed to balance the​​ partition loads, facilitating optimization​​​‌ on task-based execution environments.​ Numerical experiments are conducted​‌ on a variety of​​ test cases from electromagnetic​​​‌ simulations.

This work has​ been presented at COMPAS​‌ Conference, June 2025, Bordeaux,​​ France 46. It​​​‌ was awarded the prize​ for best poster.

8.5​‌ Optimal scheduling algorithms for​​ software-defined radio pipelined and​​ replicated task chains on​​​‌ multicore architecture (Axis 3.3.1‌)

Participants: Laércio Lima‌​‌ Pilla.

Software-Defined Radio​​ (SDR) represents a move​​​‌ from dedicated hardware to‌ software implementations of digital‌​‌ communication standards. This approach​​ offers flexibility, shorter time​​​‌ to market, maintainability, and‌ lower costs, but it‌​‌ requires an optimized distribution​​ tasks in order to​​​‌ meet performance requirements. Thus,‌ we studied the problem‌​‌ of scheduling SDR linear​​ task chains of stateless​​​‌ and stateful tasks for‌ streaming processing. We modeled‌​‌ this problem as a​​ pipelined workflow scheduling problem​​​‌ based on pipelined and‌ replicated parallelism on homogeneous‌​‌ resources. We proposed an​​ optimal dynamic programming solution​​​‌ and an optimal greedy‌ algorithm named OTAC for‌​‌ maximizing throughput while also​​ minimizing resource utilization. Moreover,​​​‌ the optimality of the‌ proposed scheduling algorithm was‌​‌ proved. We evaluated our​​ solutions and compared their​​​‌ execution times and schedules‌ to other algorithms using‌​‌ synthetic task chains and​​ an implementation of the​​​‌ DVB-S2 communication standard on‌ the AFF3CT SDR Domain‌​‌ Specific Language. Our results​​ demonstrated how OTAC quickly​​​‌ finds optimal schedules, leading‌ consistently to better results‌​‌ than other algorithms, or​​ equivalent results with fewer​​​‌ resources.

This paper has‌ been published in the‌​‌ Journal of Parallel and​​ Distributed Computing in October​​​‌ 2025 13.

8.6‌ Task-Based HPC in the‌​‌ Cloud: Price-Performance Analysis of​​ N-Body Simulations with StarPU​​​‌ (Axis 3.3.1)

Participants:‌ Laércio Lima Pilla.‌​‌

Public cloud environments present​​ significant challenges for traditional​​​‌ High Performance Computing (HPC)‌ applications due to infrastructure‌​‌ limitations that differ substantially​​ from dedicated HPC systems.​​​‌ Unlike traditional HPC clusters‌ optimized for tightly coupled‌​‌ parallel workloads, cloud platforms​​ were designed primarily for​​​‌ web services and data‌ processing applications. Key obstacles‌​‌ include high-latency networks, hardware​​ virtualization overhead, and limited​​​‌ availability of specialized accelerators,‌ all of which can‌​‌ severely impact the performance​​ of compute-intensive applications such​​​‌ as physics simulations. This‌ study investigated the feasibility‌​‌ of running HPC workloads​​ on public cloud infrastructure​​​‌ using standard and cost-effective‌ instance configurations rather than‌​‌ expensive specialized “HPC” offerings.​​ We deployed heterogeneous clusters​​​‌ on Amazon Web Services‌ using the HPC@Cloud Toolkit,‌​‌ incorporating various instance types,​​ including GPU-accelerated nodes with​​​‌ different computational capabilities. Our‌ evaluation focused on N-body‌​‌ simulations implemented using a​​ task-based parallel programming model,​​​‌ leveraging the StarPU runtime‌ system to dynamically schedule‌​‌ computational tasks across various​​ processing units. Our experimental​​​‌ results demonstrated three key‌ findings: (1) smaller GPU-equipped‌​‌ instances (g6.2xlarge)​​ achieve performance comparable to​​​‌ larger instances while costing‌ approximately one-sixth the price,‌​‌ challenging conventional scaling assumptions​​ for cloud-based HPC; (2)​​​‌ strategic GPU utilization yields‌ up to $8.‌‌2\times$ performance improvements​​ over CPU-only configurations while​​​‌ reducing total execution costs‌ by $24.\mathrm{4‌‌}\times$; and (3)​​ while task-based programming models​​​‌ effectively address network limitations‌ through dynamic scheduling, complex‌​‌ tree-based algorithms like TBFMM​​ face significant optimization challenges​​​‌ in cloud environments due‌ to load balancing issues‌​‌ and expensive parameter tuning​​ requirements. These findings provide​​​‌ practical guidance for researchers‌ and practitioners seeking cost-effective‌​‌ cloud HPC deployments, demonstrating​​​‌ that commodity cloud infrastructures​ can be viable for​‌ regular computational workloads but​​ require careful algorithmic-resource matching​​​‌ for optimal efficiency.

This​ work has been published​‌ in IEEE International Conference​​ on Cloud Engineering, September​​​‌ 2025, Rennes, France 25​.

8.7 Task-Based HPC​‌ in the Cloud: Price-Performance​​ Analysis of N-Body Simulations​​​‌ with StarPU (Topic 3.4.2​)

Participants: Laércio Lima​‌ Pilla.

Tensor-train (TT)​​ decomposition has garnered tremendous​​​‌ popularity for its efficiency​ in handling high-dimensional data​‌ arising in scientific and​​ quantum computing as well​​​‌ as machine learning applications.​ It provides a compact​‌ representation for matrices and​​ vectors with a Kronecker​​​‌ product-like low-rank structure and​ enables efficient matrix-vector operations​‌ in this compressed form.​​ The vector scalar product​​​‌ is among such key​ operations, comprising a series​‌ of tensor contractions in​​ a specific tensor network​​​‌ topology whose order significantly​ impacts the computational cost.​‌ In this work, we​​ proposed efficient algorithms for​​​‌ finding near-optimal contraction orderings​ for tensor networks representing​‌ scalar products in the​​ TT format. We showed​​​‌ that our algorithms outperform​ all existing contraction ordering​‌ methods for general tensor​​ networks where the best​​​‌ existing method incurs up​ to 15% higher cost​‌ for $x^T\mathrm{y}$, twice the cost​​​‌ for $x^T\mathrm{A}y$, and ten​‌ times higher cost for​​ $x^{T}A\mathrm{B‌}y$ scalar products where​ $x,y$ and​‌ $A,B$ are​​ vectors and matrices expressed​​​‌ in the TT format,​ respectively.

This work has​‌ been published in the​​ European Conference on Parallel​​​‌ Processing, August 2025, Dresden,​ Germany 24.

8.8​‌ MetaCS-FL: A Metaheuristic-Based Framework​​ for Client Selection in​​​‌ Federated Learning Systems (Topic​ 3.4.6)

Participants: Alan​‌ Lira Nunes, Laércio​​ Lima Pilla.

Federated​​​‌ Learning (FL) enables the​ collaborative training of distributed​‌ machine learning models, with​​ each participant (client) using​​​‌ their own local private​ data. In Cross-Device FL​‌ systems, clients usually include​​ unreliable and heterogeneous mobile​​​‌ and edge devices with​ highly imbalanced and small​‌ local datasets. Given these​​ characteristics, the selection of​​​‌ clients to participate in​ the training plays an​‌ essential role in the​​ efficacy of these systems,​​​‌ as a poor selection​ can lead to long​‌ execution times, high energy​​ consumption, and low accuracy.​​​‌ In this work, we​ proposed MetaCS-FL, a​‌ client selection framework built​​ to support different metaheuristics,​​​‌ initial solution methods, and​ user-defined triggers for new​‌ client selections. It also​​ employs client profiling and​​​‌ historical and current performance​ data to produce more​‌ efficient selections of clients​​ and the volume of​​​‌ data they should use​ for training locally. We​‌ evaluated our framework in​​ an extensive series of​​​‌ experiments, including comparisons with​ state-of-the-art algorithms, revealing the​‌ effectiveness of our approach.​​ Having FedAvg as the​​​‌ baseline for comparisons, MetaCS-FL​ reduced total time (resp.​‌ energy consumption), by up​​ to 64.83% (resp. 56.79%)​​​‌ for CIFAR-10, and by​ up to 67.59% (resp.​‌ 60.87%) for Fashion-MNIST while​​ reaching the target testing​​​‌ accuracy.

This report has​ been published in HAL​‌ in July 2025 40​​, and its paper​​ is currently under evaluation.​​​‌

8.9 Approximation Algorithms for‌ Scheduling With/Without Deadline Constraints‌​‌ Where Rejection Costs are​​ Proportional to Processing Times​​​‌ (Axis 3.3.3)

Participants:‌ Olivier Beaumont, Lionel‌​‌ Eyraud-Dubois, Laércio Lima​​ Pilla.

We studied​​​‌ two offline job scheduling‌ problems where tasks can‌​‌ be processed on a​​ limited number of energy-efficient​​​‌ edge machines or offloaded‌ to an unlimited supply‌​‌ of energy-inefficient cloud machines​​ (called rejected). The objective​​​‌ was to minimize total‌ energy consumption. First, we‌​‌ considered scheduling without deadlines,​​ formulating it as a​​​‌ scheduling problem with rejection,‌ where rejection costs are‌​‌ proportional to processing times.​​ We proposed a novel​​​‌ $\frac{5}{4}(\mathrm{1‌}+ϵ)$-approximation‌​‌ algorithm, BEKP by associating​​ it to a Multiple​​​‌ Subset Sum problem, improving‌ upon the existing $(‌‌\frac{3}{2}-\frac{\mathrm{1}}{2m})$-approximation​​​‌ for arbitrary rejection costs.‌ Next, we addressed scheduling‌​‌ with deadlines, aiming to​​ minimize the weighted number​​​‌ of rejected jobs. We‌ positioned this problem within‌​‌ the literature and introduced​​ a new $(\mathrm{1‌}-\frac{{(m-‌1)}^m}{{\mathrm{m‌‌}}^m})$-approximation algorithm,​​ MDP, inspired by an​​​‌ interval selection algorithm with‌ a $(1-‌‌\frac{m^m}{{(\mathrm{m}+1)}^{\mathrm{m‌}}})$-approximation for arbitrary‌ rejection costs. Experimental results‌​‌ demonstrate that BEKP and​​ MDP obtain better results​​​‌ (lower costs or higher‌ profits) than other state-of-the-art‌​‌ algorithms while maintaining a​​ competitive or better time​​​‌ complexity.

This work was‌ developed in the context‌​‌ of the Challenge PULSE,​​ and the paper has​​​‌ been published in IEEE‌ Transactions on Parallel and‌​‌ Distributed Systems in December​​ 2025 9.

8.10​​​‌ Energy-Aware Scheduling Strategies for‌ Partially-Replicable Task Chains on‌​‌ Heterogeneous Processors (Axis 3.3.3​​)

Participants: Laércio Lima​​​‌ Pilla.

The arrival‌ of heterogeneous (or hybrid)‌​‌ multicore architectures has brought​​ new performance trade-offs for​​​‌ applications, and efficiency opportunities‌ to systems. They have‌​‌ also increased the challenges​​ related to thread scheduling,​​​‌ as tasks' execution times‌ will vary depending if‌​‌ they are placed on​​ big (performance) cores or​​​‌ little (efficient) ones. In‌ this work, we focused‌​‌ on the challenges heterogeneous​​ multicore processors bring to​​​‌ partially-replicable task chains, such‌ as the ones that‌​‌ implement digital communication standards​​ in Software-Defined Radio (SDR).​​​‌ Our objective was to‌ maximize the throughput of‌​‌ these task chains while​​ also minimizing their power​​​‌ consumption. We modeled this‌ problem as a pipelined‌​‌ workflow scheduling problem using​​ pipelined and replicated parallelism​​​‌ on two types of‌ resources whose objectives were‌​‌ to minimize the period​​ and to use as​​​‌ many little cores as‌ necessary. We proposed two‌​‌ greedy heuristics (FERTAC and​​ 2CATAC) and one optimal​​​‌ dynamic programming (HeRAD) solution‌ to the problem. We‌​‌ evaluated our solutions and​​ compared the quality of​​​‌ their schedules (in period‌ and resource utilization) and‌​‌ their execution times using​​ synthetic task chains. We​​​‌ also studied an open‌ source implementation of the‌​‌ DVB-S2 communication standard based​​ on the StreamPU runtime.​​​‌ Leading processor vendors were‌ covered with ARM, Apple,‌​‌ AMD, and Intel platforms.​​​‌ Both the achieved throughput​ and the energy consumption​‌ were evaluated. Our results​​ demonstrated the benefits and​​​‌ drawbacks of the different​ proposed solutions.

This work​‌ has been published in​​ Heterogeneity in Computing Workshop,​​​‌ June 2025, Milan, Italy​ 22, and its​‌ extended version 39 is​​ currently under evaluation.

8.11​​​‌ HiRemate: Hierarchical Approach for​ Efficient Re-materialization of Large​‌ Neural Networks (Domain 4.2​​)

Participants: Olivier Beaumont​​​‌, Lionel Eyraud Dubois​, Yulia Gusak.​‌

Training modern neural networks​​ poses a significant memory​​​‌ challenge, as storing intermediate​ results during the forward​‌ and backward passes requires​​ considerable memory resources. To​​​‌ address this issue without​ affecting model accuracy, re-materialization​‌ techniques have been introduced​​ to recompute selected intermediate​​​‌ results instead of storing​ them, thus fulfilling the​‌ memory size constraint. The​​ main algorithmic problem is​​​‌ to compute a re-materialization​ schedule that minimizes the​‌ computational overhead within a​​ given memory budget. Our​​​‌ proposed HiRemate framework is​ based on a new​‌ hierarchical approach that provides​​ generality and quality: we​​​‌ can handle any class​ of network graphs and​‌ satisfy the memory constraint​​ with a low computational​​​‌ overhead during training. The​ framework exhibits low algorithmic​‌ complexity, making it possible​​ to scale up and​​​‌ handle very large models.​ The framework automatically builds​‌ a dataflow graph from​​ a PyTorch model, decomposes​​​‌ the graph hierarchically, and​ then builds an nn.Module​‌ that executes forward and​​ backward passes within the​​​‌ given memory budget.

This​ work has been published​‌ in the Forty-Second International​​ Conference on Machine Learning​​​‌ (ICML 2025), July 2025,​ Vancouver, Canada 19.​‌

8.12 Fault-tolerant numerical iterative​​ algorithms at scale (Topic​​​‌ 3.4.7)

Participants: Thomas​ Herault.

This year,​‌ we developed a coherent​​ set of models and​​​‌ strategies to make large-scale​ iterative computations —- with​‌ a strong emphasis on​​ linear algebra kernels and​​​‌ solvers —- both more​ resilient to errors and​‌ more efficient in their​​ use of communication and​​​‌ storage. A first contribution​ revisits protection against silent​‌ data corruptions (SDCs), where​​ errors may remain undetected​​​‌ for several iterations. Instead​ of relying on costly​‌ full replication, we studied​​ the use of partial​​​‌ detectors whose detection latency​ is bounded, and we​‌ derived optimal execution schemes​​ (segment lengths, and how​​​‌ many in-memory checkpoints must​ be kept) that guarantee​‌ correctness while reducing overhead.​​ The analysis and Monte-Carlo​​​‌ results show that, across​ a broad range of​‌ parameters, partial detection can​​ significantly outperform replication, sometimes​​​‌ yielding substantial speedups for​ realistic error rates 14​‌.

8.13 Partial Detectors​​ Versus Replication To Cope​​​‌ With Silent Errors (Axis​ 3.3.4)

Participants: Thomas​‌ Herault.

We proposed​​ a holistic fault-tolerance methodology​​​‌ for numerical iterative algorithms​ that jointly addresses the​‌ three dominant error sources​​ at scale: fail-stop failures,​​​‌ computation silent errors, and​ memory bit flips. The​‌ key idea is a​​ hierarchical periodic pattern that​​​‌ interleaves mechanisms at different​ frequencies —- (i) frequent​‌ computation verifications (“chunks”), (ii)​​ less frequent memory verification​​​‌ + in-memory checkpoints (“segments”),​ and (iii) even less​‌ frequent global checkpoints to​​ tolerate fail-stop failures (“patterns”)​​ —- and we provide​​​‌ an analytical framework to‌ derive the optimal pattern‌​‌ minimizing the expected time​​ per iteration. We instantiated​​​‌ and evaluated this approach‌ on Preconditioned Conjugate Gradient,‌​‌ illustrating scenarios where the​​ optimal pattern can dramatically​​​‌ reduce resilience overheads compared‌ to more naïve strategies‌​‌ 16, 38.​​

8.14 Fixed-Work vs. Fixed-Time​​​‌ Checkpointing on Large-Scale Failure-Prone‌ Platforms (Axis 3.3.4)‌​‌

Participants: Thomas Herault.​​

We addressed a very​​​‌ practical systems constraint that‌ directly impacts large-scale linear‌​‌ algebra runs: the prevalence​​ of fixed-length reservations on​​​‌ HPC systems. We studied‌ checkpointing not only in‌​‌ the classical “fixed-work” setting,​​ but also in the​​​‌ dual fixed-time setting, where‌ the goal is to‌​‌ maximize the expected progress​​ achieved within a reservation.​​​‌ We show that fixed-time‌ checkpointing is surprisingly harder‌​‌ than fixed-work checkpointing, propose​​ dynamic threshold-based heuristics that​​​‌ perform well for short/medium‌ reservations, and derive an‌​‌ (discretized) optimal dynamic-programming strategy,​​ including extensions to stochastic​​​‌ checkpoint durations. These results‌ provide actionable guidance for‌​‌ running iterative solvers robustly​​ under real scheduling constraints​​​‌ 8.

8.15 PaRSEC:‌ Scalability, flexibility, and hybrid‌​‌ architecture support for task-based​​ applications in ECP (Axis​​​‌ 3.3.1)

Participants: Thomas‌ Herault.

The work‌​‌ conducted during the Exascale​​ Computing Project (ECP) provided​​​‌ several key lessons on‌ the role and design‌​‌ of task-based runtime systems​​ for future large-scale platforms.​​​‌ In particular, ECP confirmed‌ that data movement, rather‌​‌ than raw computation, is​​ the dominant performance limiter​​​‌ on heterogeneous and accelerated‌ systems, making it essential‌​‌ for the runtime to​​ manage communication, data placement,​​​‌ and the overlap of‌ computation and transfers. The‌​‌ diversity of ECP target​​ architectures also showed that​​​‌ performance portability cannot be‌ achieved through static programming‌​‌ models, but instead requires​​ runtimes that dynamically adapt​​​‌ scheduling, task granularity, and‌ resource usage based on‌​‌ runtime information. In addition,​​ the coexistence of legacy​​​‌ MPI-based components with task-based‌ execution emphasized the importance‌​‌ of interoperability, while the​​ scale and duration of​​​‌ ECP runs highlighted the‌ need to treat resilience‌​‌ as a first-class runtime​​ concern, naturally enabled by​​​‌ dataflow-based execution models. These‌ lessons directly inform the‌​‌ objectives of the NumPEx​​ project, the French counterpart​​​‌ to ECP, in which‌ the TOPAL team is‌​‌ actively involved. By building​​ on the experience gained​​​‌ in ECP, our participation‌ in NumPEx aims to‌​‌ transfer and extend these​​ concepts to the French​​​‌ exascale ecosystem, contributing runtime-level‌ solutions for communication efficiency,‌​‌ adaptability, and fault tolerance​​ on next-generation European supercomputing​​​‌ platforms 10.

8.16‌ Tensor Contractions on Top‌​‌ of Runtime Systems: Application​​ to the Coupled-Cluster Method​​​‌ (Topic 3.4.1)

Participants:‌ Thomas Herault.

This‌​‌ year, we investigated how​​ the benefits of task-based​​​‌ and distributed runtime systems,‌ well established for dense‌​‌ linear algebra, can be​​ extended to tensor computations,​​​‌ which play a central‌ role in modern high-performance‌​‌ computing. Our work focused​​ on tensor contractions arising​​​‌ in computational quantum chemistry,‌ in particular in coupled-cluster‌​‌ methods, where tensors have​​ a small number of​​​‌ dimensions but very large‌ sizes. We extended the‌​‌ Chameleon dense linear algebra​​​‌ library to support tensor​ contractions by expressing them​‌ as sequences of optimized​​ matrix operations, combined with​​​‌ flexible tensor permutations. To​ address the challenges of​‌ data layout and memory​​ footprint, we identified a​​​‌ set of elementary and​ composable tensor transformations and​‌ implemented them on top​​ of the StarPU runtime​​​‌ system. We validated this​ approach on the computation​‌ of coupled-cluster residuals with​​ density fitting, demonstrating both​​​‌ its efficiency and its​ scalability on modern heterogeneous​‌ platforms 28.

8.17​​ Scalable Block-Sparse Matrix Multiplication​​​‌ Using Template Task Graphs​ (Topic 3.4.1)

Participants:​‌ Thomas Herault.

This​​ year, we advanced the​​​‌ use of task-based runtime​ systems for sparse linear​‌ algebra by addressing communication​​ scalability issues in distributed​​​‌ block-sparse matrix multiplication. Building​ on the Template Task​‌ Graph (TTG) programming model,​​ we introduced application-defined scheduling​​​‌ constraints that allow the​ runtime to control task​‌ eligibility without resorting to​​ ad-hoc control flow. Applied​​​‌ to block-sparse matrix multiplication,​ these constraints make it​‌ possible to throttle and​​ structure communication, limiting the​​​‌ number of concurrent broadcasts​ and reducing network contention​‌ while preserving overlap between​​ communication and computation. Experimental​​​‌ results demonstrate that this​ approach significantly improves scalability​‌ on large problem sizes​​ and highlights the importance​​​‌ of exposing high-level execution​ constraints to the runtime.​‌ This work reinforces the​​ role of expressive task-based​​​‌ runtimes as a key​ enabler for scalable and​‌ communication-efficient linear algebra on​​ modern distributed systems 23​​​‌.

8.18 Comparing and​ Contrasting User and Runtime​‌ Directed Data Placement Strategies​​ for Owner-Compute, Multi-accelerator Distributed​​​‌ Task Based Scheduling (Topic​ 3.4.1)

Participants: Thomas​‌ Herault.

This work​​ explores data placement strategies​​​‌ in task-based runtime systems​ for linear algebra applications​‌ on multi-accelerator, distributed platforms.​​ Using the PaRSEC runtime,​​​‌ we compared runtime-directed heuristics​ and user-directed placement strategies​‌ in the context of​​ owner-compute scheduling, focusing on​​​‌ dense matrix multiplication and​ Cholesky factorization. The results​‌ show that while automated​​ strategies can significantly improve​​​‌ locality and outperform naïve​ approaches, they remain consistently​‌ outperformed by carefully designed​​ user-directed placements, particularly at​​​‌ scale. The study also​ highlights the limitations of​‌ relying on unified virtual​​ memory and demonstrates the​​​‌ importance of explicitly managing​ data received from the​‌ network, especially on modern​​ systems where network interfaces​​​‌ are directly attached to​ accelerators. Overall, this work​‌ emphasizes that runtime systems​​ must expose flexible mechanisms​​​‌ for expressing data placement​ policies, allowing expert users​‌ to guide execution while​​ preserving a clear separation​​​‌ between algorithmic expression and​ performance tuning 17.​‌

8.19 Optimizing Parallel Heterogeneous​​ System Efficiency: Dynamic Task​​​‌ Graph Adaptation with Recursive​ Tasks (Topic 3.4.1)​‌

Participants: Abdou Guermouche.​​

Task-based programming models are​​​‌ currently an ample trend​ to leverage heterogeneous parallel​‌ systems in a productive​​ way (OpenACC, Kokkos, Legion,​​​‌ OmpSs, PaRSEC, StarPU, XKaapi,​ ...). Among these models,​‌ the Sequential Task Flow​​ (STF) model is widely​​​‌ embraced (PaRSEC's DTD, OmpSs,​ StarPU) since it allows​‌ to express task graphs​​ naturally through a sequential-looking​​​‌ submission of tasks, and​ tasks dependencies are inferred​‌ automatically. However, STF is​​ limited to task graphs​​ with task sizes that​​​‌ are fixed at submission,‌ posing a challenge in‌​‌ determining the optimal task​​ granularity. Notably, in heterogeneous​​​‌ systems, the optimal task‌ size varies across different‌​‌ processing units, so a​​ single task size would​​​‌ not fit all units.‌ StarPU's recursive tasks allow‌​‌ graphs with several task​​ granularities by turning some​​​‌ tasks into sub-graphs dynamically‌ at runtime. The decision‌​‌ to transform these tasks​​ into sub-graphs is decided​​​‌ by a StarPU component‌ called the Splitter. After‌​‌ deciding to transform some​​ tasks, classical scheduling approaches​​​‌ are used, making this‌ component generic, and orthogonal‌​‌ to the scheduler. In​​ this paper, we propose​​​‌ a new policy for‌ the Splitter, which is‌​‌ designed for heterogeneous platforms,​​ that relies on linear​​​‌ programming aimed at minimizing‌ execution time and maximizing‌​‌ resource utilization. This results​​ in a dynamic well-balanced​​​‌ set comprising both small‌ tasks to fill multiple‌​‌ CPU cores, and large​​ tasks for efficient execution​​​‌ on accelerators like GPU‌ devices. We then present‌​‌ an experimental evaluation showing​​ that just-in-time adaptations of​​​‌ the task graph lead‌ to improved performance across‌​‌ various dense linear algebra​​ algorithms 12.

8.20​​​‌ Improving energy efficiency of‌ HPC applications using unbalanced‌​‌ GPU power capping (Topic​​ 3.4.3)

Participants: Abdou​​​‌ Guermouche, Hayfa Tayeb‌.

Energy efficiency represents‌​‌ a significant challenge in​​ the domain of High-Performance​​​‌ Computing (HPC). One potential‌ key parameter to improve‌​‌ energy efficiency is the​​ use of power capping,​​​‌ a technique for controlling‌ the power limits of‌​‌ a device, such as​​ a CPU or GPU.In​​​‌ this paper, we propose‌ to examine the impact‌​‌ of GPU power capping​​ in the context of​​​‌ HPC applications using heterogeneous‌ computing systems. To this‌​‌ end, we first conduct​​ an extensive study of​​​‌ the impact of GPU‌ power capping on a‌​‌ compute intensive kernel, namely​​ matrix multiplication kernel (GEMM),​​​‌ on different Nvidia GPU‌ architectures. Interestingly, such compute-intensive‌​‌ kernels are up to​​ 30 % more energy​​​‌ efficient when the GPU‌ is set to 55-70‌​‌ % of its Thermal​​ Design Power (TDP). Using​​​‌ the best power capping‌ configuration provided by this‌​‌ study, we investigate how​​ setting different power caps​​​‌ for GPU devices of‌ a heterogeneous computing node‌​‌ can improve the energy​​ efficiency of the running​​​‌ application. We consider dense‌ linear algebra task-based operations,‌​‌ namely matrix multiplication and​​ Cholesky factorization.We show how​​​‌ the underlying runtime system‌ scheduler can then automatically‌​‌ adapt its decisions to​​ take advantage of the​​​‌ heterogeneous performance capability of‌ each GPU. The results‌​‌ show that for a​​ given platform equipped with​​​‌ four GPU devices, applying‌ a power cap on‌​‌ all GPUs improves the​​ energy efficiency for matrix​​​‌ multiplication up to 24.3‌ % (resp. 33.78 %)‌​‌ for double (resp. single)​​ precision 26.

8.21​​​‌ Sparse Matrix Ordering for‌ Fine Grain Parallel Triangular‌​‌ Solve Using SIMD (Topic​​ 3.4.1)

Participants: Abdou​​​‌ Guermouche.

The evolution‌ of processor hardware increasingly‌​‌ supports fine grain parallelism​​ through SIMD (Single Instruction,​​​‌ Multiple Data) vector instruction‌ sets and hardware threading.‌​‌ For instance, the new​​​‌ ARM SVE instruction set​ allows for hardware implementation​‌ of up to 32​​ double precision SIMD vector​​​‌ sizes per hardware thread.​ In this work, we​‌ focus on vectorization of​​ the triangular solves required​​​‌ in BiCGStab preconditioned with​ ILU(0) that is particularly​‌ numerically effective for IFPEN​​ applications. In our context,​​​‌ expressing some parallelism can​ be achieved by changing​‌ the sparse structure of​​ the matrices through unknown​​​‌ ordering; that can be​ recast in terms of​‌ graph ordering and coloring.​​ We use a graph​​​‌ coloring method named ColorRCM​ to exhibit fine grain​‌ parallelism to feed the​​ SIMD computing units while​​​‌ improving the convergence of​ the Krylov solver compared​‌ to classical greedy graph​​ coloring method. We first​​​‌ evaluate the performance of​ SIMD-SpTRSV using the permutation​‌ provided by ColorRCM and​​ achieve an acceleration between​​​‌ 1.7 and 6 in​ AVX2 compared to Intel​‌ MKL 21.4. Then we​​ examine the impact of​​​‌ ColorRCM ordering on ILU(0)-BiCGStab​ performance on 201 matrices,​‌ including those from the​​ Suite Sparse matrix (The​​​‌ University of Florida Sparse​ Matrix Collection collection and​‌ from the IFPEN porous​​ media flow simulations. The​​​‌ solver configuration uses the​ ColorRCM ordering and vectorized​‌ with AVX2 instructions showed​​ the best convergence times​​​‌ in two thirds of​ the tests 21.​‌

8.22 Mind Bubbles and​​ Memory: Bounds on Scheduling​​​‌ Pipeline Parallelism with Rematerialization​ (Domain 4.2)

Participants:​‌ Adrien Aguila–Multner, Yulia​​ Gusak, Olivier Beaumont​​​‌, Lionel Eyraud Dubois​.

Training large neural​‌ networks, especially Transformer-based Large​​ Language Models (LLMs), requires​​​‌ massive high-performance computing (HPC)​ resources. Within each microbatch,​‌ computations follow a strictly​​ sequential flow through a​​​‌ stack of transformer blocks:​ a forward pass to​‌ compute the loss, and​​ a backward pass to​​​‌ propagate gradients. This sequential​ structure limits intrinsic parallelism.​‌ To improve performance, several​​ complementary strategies have been​​​‌ developed: data, tensor, sequence,​ and pipeline parallelism, typically​‌ combined to achieve scalability​​ over tens of thousands​​​‌ of GPUs.

In 35​, we present a​‌ formal analysis of pipeline​​ parallelism (PP) for large-scale​​​‌ training. In PP, the​ model is partitioned into​‌ multiple stages, and microbatches​​ are injected into the​​​‌ pipeline to overlap computation.​ The main challenge is​‌ to minimize idle periods​​ (pipeline bubbles) while managing​​​‌ memory usage, since each​ GPU must store intermediate​‌ activations from multiple in-flight​​ microbatches. Existing scheduling algorithms​​​‌ such as GPIPE, 1F1B,​ HANAYO, and MEGATRON reduce​‌ idle time but lack​​ formal lower bounds or​​​‌ explicit modeling of memory​ constraints.

We develop a​‌ unified analytical approach for​​ PP scheduling, deriving lower​​​‌ bounds on completion time​ for both single-wave and​‌ multi-wave regimes. Our analysis​​ explicitly incorporates a memory​​​‌ constraint K, denoting the​ number of activations that​‌ can be stored per​​ GPU. Exact results are​​​‌ provided for two extreme​ cases (minimal memory (​‌$K=1$)​​ and large memory (​​​‌$K\ge m$)),​ while general lower bounds​‌ are established for intermediate​​ configurations. Our analysis highlights​​​‌ the intrinsic coupling between​ pipeline utilization and memory​‌ footprint, providing a foundation​​ for evaluating and comparing​​ pipeline scheduling algorithms under​​​‌ realistic memory constraints.

8.23‌ Optimized Forward-Backward Rematerialization for‌​‌ Memory-Efficient Pipeline Parallel Training​​ (Topic 3.4.4)

Participants:​​​‌ Adrien Aguila–Multner, Olivier‌ Beaumont, Lionel Eyraud‌​‌ Dubois, Yulia Gusak​​.

Pipeline parallelism is​​​‌ a key technique for‌ scaling deep network training‌​‌ across multiple devices. Recent​​ works have significantly reduced​​​‌ pipeline idle time by‌ improving scheduling efficiency. Decoupling‌​‌ the computation of gradients​​ with respect to weights​​​‌ and activations led to‌ the development of schedules‌​‌ with almost no idle​​ time. However, these methods​​​‌ still require substantial memory,‌ limiting their applicability on‌​‌ resource-constrained hardware.

In 36​​, our first contribution​​​‌ is to introduce recomputation‌ to the backward pass,‌​‌ extending rematerialization beyond the​​ forward pass. This enables​​​‌ executing schedules with decoupled‌ gradient computations under much‌​‌ tighter memory constraints. Our​​ second contribution is a​​​‌ unified optimization approach that,‌ given a model and‌​‌ hardware memory constraints, formulates​​ and solves an Integer​​​‌ Linear Programming (ILP) problem‌ to determine the optimal‌​‌ per-microbatch, per-GPU rematerialization strategy​​ for a given schedule,​​​‌ applicable to both one-wave‌ and multi-wave pipeline schedules.‌​‌ Our third contribution shows​​ that, as device memory​​​‌ constraints vary, the relative‌ advantages of different pipeline‌​‌ schedules also change in​​ the presence of rematerialization.​​​‌ We provide corresponding insights‌ and a PyTorch framework‌​‌ that enables finding and​​ executing the optimal combination​​​‌ of pipeline scheduling and‌ rematerialization strategies. Experiments demonstrate‌​‌ the effectiveness of all​​ three contributions, showing that​​​‌ our approach enables efficient‌ training of larger models‌​‌ under tight memory budgets,​​ adapts optimally to varying​​​‌ memory capacities, and reduces‌ recomputation overhead compared to‌​‌ existing recomputation solutions.

8.24​​ Leveraging Expert Usage to​​​‌ Speed up LLM Inference‌ with Expert Parallelism (Topic‌​‌ 3.4.4)

Participants: Olivier​​ Beaumont, Raphael Bourgouin​​​‌.

Large language models‌ have become indispensable for‌​‌ many text-processing applications. Their​​ inference 15, i.e.​​​‌ their use to generate‌ text, is a time-consuming‌​‌ task since tokens have​​ to be generated one​​​‌ after the other, even‌ if the computational load‌​‌ has been reduced by​​ model sparsification, e.g. by​​​‌ using a Mixture of‌ Experts (MoE) models. In‌​‌ the MoE context, a​​ subset of experts is​​​‌ selected at each stage.‌ Note that not all‌​‌ subsets of experts (pairs​​ of experts in most​​​‌ cases) in a given‌ layer have the same‌​‌ probability of being selected.​​ When experts are mapped​​​‌ to different GPUs, there‌ is a risk of‌​‌ load imbalance if the​​ selected experts end up​​​‌ on a small number‌ of GPUs. This paper‌​‌ proposes to leverage this​​ heterogeneity in expert usage​​​‌ to map experts of‌ popular subsets onto distinct‌​‌ GPUs, allowing them to​​ be processed in parallel​​​‌ and thus reducing the‌ time needed for inference.‌​‌ Even though this mapping​​ problem is NP-complete, it​​​‌ is possible to design‌ simple greedy strategies that‌​‌ significantly reduce the need​​ for sequential expert processing.​​​‌ Our proof-ofconcept confirms that‌ our mapping strategies effectively‌​‌ reduce inference time on​​ the Mixtral model.

8.25​​​‌ Pallas: a generic‌ trace format for large‌​‌ HPC trace analysis (Axis​​​‌ 3.3.1)

Participant: Philippe​ Swartvagher.

Identifying performance​‌ bottlenecks in a parallel​​ application is tedious, especially​​​‌ because it requires analyzing​ the behaviour of various​‌ software components, as bottlenecks​​ may have several causes​​​‌ and symptoms. For example,​ a load imbalance may​‌ cause long MPI waiting​​ times, or contention on​​​‌ disk may degrade the​ performance of I/O operations.​‌ Detecting a performance problem​​ means investigating the execution​​​‌ of an application and​ applying several performance analysis​‌ techniques. To do so,​​ one can use a​​​‌ tracing tool to collect​ information describing the behaviour​‌ of the application. At​​ the end of the​​​‌ execution, a trace file​ in a specific format​‌ is available to the​​ application user, which can​​​‌ be used to conduct​ a complete post-mortem investigation.​‌ Several challenges emerge from​​ the generation and use​​​‌ of traces. Tracing applications​ may alter the performance​‌ of the application, and​​ can create thousands of​​​‌ heavy trace files, especially​ at a large scale.​‌ Most importantly, the post-mortem​​ analysis needs to load​​​‌ these thousands of trace​ files in memory, and​‌ process them. This quickly​​ becomes impractical for large​​​‌ scale applications, as memory​ gets exhausted and the​‌ number of opened files​​ exceeds the system capacity.​​​‌ In this paper, we​ propose Pallas 18,​‌ a generic trace format​​ tailored for conducting various​​​‌ post-mortem performance analysis of​ traces describing large executions​‌ of HPC applications. During​​ the execution of the​​​‌ application, Pallas collects events​ and detects their repetitions​‌ on-the-fly. When storing the​​ trace to disk, Pallas​​​‌ groups the data from​ similar events or groups​‌ of events together in​​ order to later speed​​​‌ up trace reading. We​ demonstrate that the Pallas​‌ online detection of the​​ program structure does not​​​‌ significantly degrade the performance​ of the applications. Moreover,​‌ the Pallas format allows​​ faster trace analysis compared​​​‌ to other evaluated trace​ formats. Overall, the Pallas​‌ trace format allows an​​ interactive analysis of a​​​‌ trace that is required​ when a user investigates​‌ a performance problem.

9.1 Bilateral​ Grants with Industry

Participants:​‌ Olivier Beaumont, Lionel​​ Eyraud-Dubois, Mathieu Faverge​​​‌, Abdou Guermouche,​ Yulia Gusak, Pierre​‌ Ramet.

Some on​​ the ongoing PhD thesis​​​‌ are developed within bilateral​ contract with industry for​‌ PhD advisory:

• Airbus (2022-).​​ This collaboration concerns the​​​‌ parallelization and optimization of​ the Flusepa application, which​‌ models the separation of​​ boosters for space launchers​​​‌ at Airbus Safran Launchers.​ Flusepa combines computational fluid​‌ mechanics, algorithms (AMR) and​​ task-based parallelism based on​​​‌ the StarPU runtime system.​ We are involved in​‌ the supervision of the​​ PhD. of Alice Lasserre​​​‌ in this context.
• CEA-Cesta​ for the PhD of​‌ Abel Calluaud. A direct​​ solver developed at CEA​​​‌ relies on the approximation​ by hierarchical matrices to​‌ reduce both computational and​​ memory costs. Although these​​​‌ developments have met a​ growing demand for increased​‌ simulation accuracy, there are​​ still open problems to​​​‌ pursue these research efforts​ in an HPC context.​‌ In this thesis, we​​ propose to develop and​​ compare several approaches to​​​‌ adapt the granularity of‌ hierarchical tasks and extract‌​‌ parallelism to exploit the​​ multicore computational nodes associated​​​‌ with massively parallel architectures‌ such as GPUs.
• CEA-Cesta‌​‌ for the PhD of​​ Dimitri Walther. In the​​​‌ context of numerical simulation‌ of electromagnetism, integral methods‌​‌ are among the most​​ widely used because of​​​‌ their power. These methods‌ lead to the solution‌​‌ of dense linear problems​​ and are therefore very​​​‌ expensive. For this reason,‌ hierarchical compression methods have‌​‌ been developed that drastically​​ reduce the cost associated​​​‌ with these matrices. They‌ are based on a‌​‌ hierarchical partitioning of the​​ matrix, and therefore of​​​‌ the mesh, and the‌ efficiency of the compression‌​‌ depends on this partitioning.​​ In this context, the​​​‌ aim of the thesis‌ is to develop efficient‌​‌ and scalable hierarchical partitioners​​ to optimise the compression​​​‌ of the matrix.
• Eviden‌ for the PhD of‌​‌ Alycia Lisito. For over​​ three years, we have​​​‌ been collaborating with Eviden‌ on the development of‌​‌ an HPL benchmark on​​ top of runtime systems.​​​‌ This work is continued‌ as part of Alycia‌​‌ Lisito's thesis funded by​​ a CIFRE contract. To​​​‌ guarantee a high level‌ of flexibility and portability,‌​‌ it is possible to​​ use a task-based implementation​​​‌ through an executive support‌ (or runtime). This programming‌​‌ model has already proved​​ its effectiveness in the​​​‌ implementation of various parallel‌ algorithms, in particular for‌​‌ dense linear algebra (LU​​ decomposition, Cholesky decomposition, QR,​​​‌ etc.). In this thesis,‌ we will use Inria's‌​‌ existing software stack, through​​ the dense linear algebra​​​‌ library Chameleon and the‌ executive support StarPU. These‌​‌ reference libraries for runtime​​ linear algebra will be​​​‌ studied to enable the‌ scaling up of more‌​‌ complex algorithms such as​​ HPL.
• Eviden for the​​​‌ PhD of Jean Conan.‌ Within the framework of‌​‌ High-Performance Computing (HPC) tenders,​​ Atos Bull must provide​​​‌ contractual performance guarantees for‌ future supercomputers. However, direct‌​‌ measurement is often impossible​​ during the bidding phase,​​​‌ either because the hardware‌ components (processors, accelerators) are‌​‌ not yet commercially available​​ or because the scale​​​‌ of the proposed system‌ exceeds the testing resources‌​‌ available internally. Performance prediction​​ has become a critical​​​‌ tool, not only for‌ meeting client requirements but‌​‌ also for upstream architecture​​ sizing (such as network​​​‌ topology) and optimizing massively‌ parallel software. The transition‌​‌ to exascale computing introduces​​ unprecedented complexity, driven by​​​‌ the increasing heterogeneity of‌ compute nodes and the‌​‌ intricate structure of high-speed​​ networks. The objective of​​​‌ the thesis is thus‌ to explore novel methodologies‌​‌ for performance prediction, with​​ a primary focus on​​​‌ simulation techniques, Reduce optimization‌ overhead by minimizing the‌​‌ number of large-scale physical​​ runs required to determine​​​‌ optimal execution parameters, and‌ finally accurately model the‌​‌ impact of node heterogeneity​​ and network architecture on​​​‌ overall system performance.
• Diabolocom‌ and Inria are now‌​‌ on the final stage​​ of the contract negotiation​​​‌ to start a PhD‌ thesis co-supervised by Yulia‌​‌ Gusak and Olivier Beaumont​​ on Optimization of Multi-Stage​​​‌ Generative Model Pipelines for‌ Cost-Efficient, Scalable Inference.

10.1​​​‌ International initiatives

10.2 International research​ visitors

10.3 European initiatives​​

10.4 National initiatives​‌

10.5 Public policy support​‌

Olivier Beaumont conducted an​​ expert assessment, in collaboration​​​‌ with IRD and other​ Inria colleagues, on the​‌ current state and future​​ development prospects of high-performance​​​‌ computing in Africa. This​ study was commissioned by​‌ the French Development Agency​​ (Agence Française de​​​‌ Développement).

11.1 Promoting scientific activities​‌

11.2 Teaching - Supervision​​​‌ - Juries - Educational​ and pedagogical outreach

• Undergraduate​‌ level/Licence:
  • Aurélien Esnard :​​ Network (54h), Software technologies​​​‌ (80h) at Bordeaux University.​
  • Pierre Ramet : System​‌ programming 24h, Databases 32h,​​ Object programming 48h, Distributed​​​‌ programming 16h, Cryptography 16h,​ Introduction to unsupervised learning​‌ 16h at Bordeaux University.​​
  • Philippe Swartvagher : C​​​‌ Programming (46h), Web Programming​ (36h), Tools for Programming​‌ and C project (30h)​​ at Bordeaux INP (​​​‌Enseirb-MatMeca).
  • Abdou Guermouche​ System programming 36h at​‌ Bordeaux University.
  • Mathieu Faverge​​ : Programming environment (26h),​​​‌ Numerical algorithmic (25h), C​ projects (25h) at Bordeaux​‌ INP (Enseirb-MatMeca).​​
• Post graduate level/Master:
  • Aurélien​​​‌ Esnard : Network management​ (24h), Network security (24h)​‌ at Bordeaux University.
  • Lionel​​ Eyraud Dubois : Graphs​​​‌ and Algorithms (20h), Complexity​ and Approximation (20h) at​‌ Bordeaux University.
  • Olivier Beaumont​​ : Parallel Algorithms, 20h​​​‌ at Bordeaux INP.
  • Pierre​ Ramet : Cryptography 20h​‌ and Numerical algorithms 40h​​ at Bordeaux INP (​​​‌Enseirb-MatMeca).
  • Philippe Swartvagher​ : Parallel Algorithms (17h),​‌ Project of network and​​ system programming (25h), Operating​​ Systems (15h) at Bordeaux​​​‌ INP (Enseirb-MatMeca).‌
  • Abdou Guermouche Network management‌​‌ 92h, Network security 64h,​​ Operating system 24h at​​​‌ Bordeaux University.
  • Mathieu Faverge‌ : System programming: lecture,‌​‌ practice and project (54h),​​ Linear Algebra for high​​​‌ Performance Computing (9h) at‌ Bordeaux INP (Enseirb-MatMeca‌​‌). He is also​​ in charge of the​​​‌ master 2 internship for‌ the Computer Science department‌​‌ at Bordeaux INP (Enseirb-MatMeca)​​ and he is in​​​‌ charge, with Abdou Guermouche‌ , of the High‌​‌ Performance Computing - High​​ Performance Data Analytics specialty​​​‌ at Enseirb-MatMeca. This is‌ a common training curriculum‌​‌ between the Computer Science​​ and the MatMeca departments​​​‌ at Bordeaux INP and‌ with the Bordeaux University‌​‌ in the context of​​ the Computer Science Research​​​‌ Master.
  • Yulia Gusak :‌ Efficient Deep Learning (Outils‌​‌ pour l'apprentissage) (19h) at​​ Bordeaux INP (Enseirb-MatMeca​​​‌).
  • Laércio Lima Pilla‌ : Algorithms for High-Performance‌​‌ Computing Platforms (16h) at​​ Bordeaux INP (Enseirb-MatMeca​​​‌) and Bordeaux University,‌ Reading articles and scientific‌​‌ documentation (3h) at Bordeaux​​ University.
  • Thomas Herault :​​​‌ Introduction to tensor algebra‌ for the Engineer in‌​‌ Computer Science (9h) at​​ Bordeaux INP (Enseirb-MatMeca​​​‌); Open MP programming‌ (8h) at Bordeaux INP‌​‌ (Enseirb-MatMeca).

11.3 Popularization

12.1​​ Major publications

• 1 inproceedings​​​‌O.Olivier Beaumont,‌ P.Philippe Duchon,‌​‌ L.Lionel Eyraud-Dubois,​​ J.Julien Langou and​​​‌ M.Mathieu Vérité.‌ Symmetric Block-Cyclic Distribution: Fewer‌​‌ Communications Leads to Faster​​ Dense Cholesky Factorization.​​​‌SC 2022 - Supercomputing‌Dallas, Texas, United States‌​‌November 2022HAL
• 2​​ inproceedingsO.Olivier Beaumont​​​‌, L.Lionel Eyraud-Dubois‌, M.Mathieu Vérité‌​‌ and J.Julien Langou​​. I/O-Optimal Algorithms for​​​‌ Symmetric Linear Algebra Kernels‌.ACM Symposium on‌​‌ Parallelism in Algorithms and​​ ArchitecturesPhiladelphie, United States​​​‌July 2022HAL
• 3‌ articleR.Robert Falgout‌​‌, M.Matthieu Lecouvez​​, P.Pierre Ramet​​​‌ and C.Clément Richefort‌. Toward an Algebraic‌​‌ Multigrid Method for the​​ Indefinite Helmholtz Equation.​​​‌SIAM Journal on Scientific‌ ComputingAugust 2025,‌​‌ S285-S310HALDOI
• 4​​ articleM.Mathieu Faverge​​​‌, N.Nathalie Furmento‌, A.Abdou Guermouche‌​‌, G.Gwenolé Lucas​​, R.Raymond Namyst​​​‌, S.Samuel Thibault‌ and P.Pierre‐andré Wacrenier‌​‌. Programming Heterogeneous Architectures​​ Using Hierarchical Tasks.​​​‌Concurrency and Computation: Practice‌ and Experience3525‌​‌2023HALDOI
• 5​​ inproceedingsJ.Julia Gusak​​​‌, X.Xunyi Zhao‌, T.Théotime Le‌​‌ Hellard, Z.Zhe​​ Li, L.Lionel​​​‌ Eyraud-Dubois and O.Olivier‌ Beaumont. HiRemate: Hierarchical‌​‌ Approach for Efficient Re-materialization​​ of Large Neural Networks​​​‌.Proceedings of the‌ 42nd International Conference on‌​‌ Machine LearningForty-Second International​​ Conference on Machine Learning​​​‌ (ICML 2025)267Vancouver,‌ Canada2025HALback‌​‌ to textback to​​ text
• 6 inproceedingsA.​​​‌Alycia Lisito, M.‌Mathieu Faverge, M.‌​‌Matthieu Kuhn, F.​​Florent Pruvost and P.​​​‌Pierre Ramet. Scalable‌ and portable LU factorization‌​‌ with partial pivoting on​​ top of runtime systems​​​‌.IPDPS25 - 39th‌ IEEE International Parallel and‌​‌ Distributed Processing SymposiumMilan,​​​‌ ItalyJune 2025HAL​
• 7 inproceedingsX.Xunyi​‌ Zhao, T.Théotime​​ Le Hellard, L.​​​‌Lionel Eyraud-Dubois, J.​Julia Gusak and O.​‌Olivier Beaumont. Rockmate:​​ an Efficient, Fast, Automatic​​​‌ and Generic Tool for​ Re-materialization in PyTorch.​‌ICML 2023Honolulu (HI),​​ United StatesJuly 2023​​​‌HAL

12.2 Publications of​ the year

12.3 Cited publications

• 47​​​‌ articleE.Emmanuel Agullo​, O.Olivier Aumage​‌, M.Mathieu Faverge​​, N.Nathalie Furmento​​​‌, F.Florent Pruvost​, M.Marc Sergent​‌ and S. P.Samuel​​ Paul Thibault. Achieving​​​‌ High Performance on Supercomputers​ with a Sequential Task-based​‌ Programming Model.IEEE​​ Transactions on Parallel and​​​‌ Distributed Systems2017,​ 1-1DOIback to​‌ text
• 48 articleE.​​Emmanuel Agullo, A.​​​‌Alfredo Buttari, A.​Abdou Guermouche and F.​‌Florent Lopez. Implementing​​ Multifrontal Sparse Solvers for​​​‌ Multicore Architectures with Sequential​ Task Flow Runtime Systems​‌.ACM Trans. Math.​​ Softw.432August​​​‌ 2016, 13:1--13:22HAL​DOIback to text​‌
• 49 inproceedingsE.Emmanuel​​ Agullo, A.Alfredo​​​‌ Buttari, A.Abdou​ Guermouche and F.Florent​‌ Lopez. Task-Based Multifrontal​​ QR Solver for GPU-Accelerated​​​‌ Multicore Architectures..HiPC​Best paper awardIEEE​‌ Computer Society2015,​​ 54-63HALDOIback​​​‌ to text
• 50 inproceedings​P.Pedro Alonso,​‌ M. F.Manuel F.​​ Dolz, F. D.​​​‌Francisco D. Igual,​ R.Rafael Mayo and​‌ E. S.Enrique S.​​ Quintana-Ortí. Reducing Energy​​​‌ Consumption of Dense Linear​ Algebra Operations on Hybrid​‌ CPU-GPU Platforms.2012​​ IEEE 10th International Symposium​​​‌ on Parallel and Distributed​ Processing with Applications2012​‌, 56-62DOIback​​ to text
• 51 article​​​‌P.Pedro Alonso,​ M. F.Manuel F.​‌ Dolz, R.Rafael​​ Mayo and E. S.​​​‌Enrique S. Quintana-Ortí.​ Modeling power and energy​‌ consumption of dense matrix​​ factorizations on multicore processors​​​‌.Concurrency and Computation:​ Practice and Experience26​‌172014, 2743-2757​​URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.3162DOIback​​​‌ to text
• 52 inproceedings​H.Hartwig Anzt,​‌ J.Jack Dongarra and​​ E. S.Enrique S.​​​‌ Quintana-Ortí. Adaptive Precision​ Solvers for Sparse Linear​‌ Systems.Proceedings of​​ the 3rd International Workshop​​​‌ on Energy Efficient Supercomputing​E2SC '15New York,​‌ NY, USAAustin, Texas​​Association for Computing Machinery​​​‌2015, URL: https://doi.org/10.1145/2834800.2834802​DOIback to text​‌
• 53 inproceedingsO.Olivier​​ Beaumont, P.Philippe​​​‌ Duchon, L.Lionel​ Eyraud-Dubois, J.Julien​‌ Langou and M.Mathieu​​ Verite. Symmetric Block-Cyclic​​​‌ Distribution: Fewer Communications leads​ to Faster Dense Cholesky​‌ Factorization.SC'22: Proceedings​​ of the International Conference​​ for High Performance Computing,​​​‌ Networking, Storage and Analysis‌(best paper, Algorithm Track)‌​‌IEEE and ACM2022​​back to textback​​​‌ to text
• 54 techreport‌O.Olivier Beaumont,‌​‌ L.Lionel Eyraud-Dubois,​​ J.Julien Herrmann,​​​‌ A.Alexis Joly and‌ A.Alena Shilova.‌​‌ Optimal checkpointing for heterogeneous​​ chains: how to train​​​‌ deep neural networks with‌ limited memory.RR-9302‌​‌Inria Bordeaux Sud-OuestNovember​​ 2019HALback to​​​‌ textback to text‌
• 55 inproceedingsO.Olivier‌​‌ Beaumont, L.Lionel​​ Eyraud-Dubois and A.Alena​​​‌ Shilova. Efficient Combination‌ of Rematerialization and Offloading‌​‌ for Training DNNs.​​NeurIPS 2021 - Thirty-fifth​​​‌ Conference on Neural Information‌ Processing SystemsVirtual-only Conference‌​‌December 2021HALback​​ to textback to​​​‌ text
• 56 inproceedingsO.‌Olivier Beaumont, L.‌​‌Lionel Eyraud-Dubois and A.​​Alena Shilova. MadPipe:​​​‌ Memory Aware Dynamic Programming‌ Algorithm for Pipelined Model‌​‌ Parallelism.2022 IEEE​​ International Parallel and Distributed​​​‌ Processing Symposium Workshops (IPDPSW)‌IEEE2022back to‌​‌ text
• 57 inproceedingsO.​​Olivier Beaumont, L.​​​‌Lionel Eyraud-Dubois and A.‌Alena Shilova. Optimal‌​‌ GPU-CPU Offloading Strategies for​​ Deep Neural Network Training​​​‌.Euro-Par 2020: Parallel‌ ProcessingChamSpringer International‌​‌ Publishing2020, 151--166​​back to textback​​​‌ to text
• 58 inproceedings‌O.Olivier Beaumont,‌​‌ L.Lionel Eyraud-Dubois and​​ A.Alena Shilova.​​​‌ Pipelined Model Parallelism: Complexity‌ Results and Memory Considerations‌​‌.Proceedings of Europar​​ 2021Lisbon, PortugalSpringer​​​‌August 2021HALback‌ to textback to‌​‌ text
• 59 inproceedingsO.​​Olivier Beaumont, L.​​​‌Lionel Eyraud-Dubois and M.‌Mathieu Verite. 2D‌​‌ Static Resource Allocation for​​ Compressed Linear Algebra and​​​‌ Communication Constraints.2020‌ IEEE 27th International Conference‌​‌ on High Performance Computing,​​ Data, and Analytics (HiPC)​​​‌IEEE2020, 181--191‌back to text
• 60‌​‌ inproceedingsO.Olivier Beaumont​​, L.Lionel Eyraud-Dubois​​​‌, M.Mathieu Vérité‌ and J.Julien Langou‌​‌. I/O-Optimal Algorithms for​​ Symmetric Linear Algebra Kernels​​​‌.ACM Symposium on‌ Parallelism in Algorithms and‌​‌ ArchitecturesAssociation for Computing​​ Machinery : SIGACT, SIGARCH​​​‌Philadelphie, United StatesJuly‌ 2022HALback to‌​‌ textback to text​​
• 61 articleO.Olivier​​​‌ Beaumont, J.Julien‌ Herrmann, G.Guillaume‌​‌ Pallez and A.Alena​​ Shilova. Optimal memory-aware​​​‌ backpropagation of deep join‌ networks.Philosophical Transactions‌​‌ of the Royal Society​​ A37821662020​​​‌, 20190049back to‌ text
• 62 inproceedingsR.‌​‌Rocío Carratalá-Sáez, M.​​Mathieu Faverge, G.​​​‌Grégoire Pichon, E.‌ S.Enrique Salvador Quintana-Ortí‌​‌ and G.Guillaume Sylvand​​. Exploiting Generic Tiled​​​‌ Algorithms Toward Scalable H-Matrices‌ Factorizations on Top of‌​‌ Runtime Systems.SIAM​​ PP20-SIAM Conference on Parallel​​​‌ Processing for Scientific Computing‌2020back to text‌​‌
• 63 inproceedingsR.Rocío​​ Carratalá-Sáez, M.Mathieu​​​‌ Faverge, G.Grégoire‌ Pichon, G.Guillaume‌​‌ Sylvand and E. S.​​Enrique S Quintana-Ortí.​​​‌ Tiled Algorithms for Efficient‌ Task-Parallel ?-Matrix Solvers.‌​‌2020 IEEE International Parallel​​ and Distributed Processing Symposium​​​‌ Workshops (IPDPSW)IEEE2020‌, 757--766back to‌​‌ text
• 64 mastersthesisT.​​​‌Tanguy Chatelain. Anticipation​ des communications réseau grâce​‌ à la connaissance du​​ futur dans le parallélisme​​​‌ à tâche.MA​ ThesisEnseirb-MatmecaSeptember 2025​‌HALback to text​​
• 65 inproceedingsV.Viktoriia​​​‌ Chekalina, G.Georgiy​ Novikov, J.Julia​‌ Gusak, A.Alexander​​ Panchenko and I.Ivan​​​‌ Oseledets. Efficient gpt​ model pre-training using tensor​‌ train matrix representation.​​Proceedings of the 37th​​​‌ Pacific Asia Conference on​ Language, Information and Computation​‌2023, 600--608back​​ to text
• 66 article​​​‌T.Tianqi Chen,​ B.Bing Xu,​‌ C.Chiyuan Zhang and​​ C.Carlos Guestrin.​​​‌ Training deep nets with​ sublinear memory cost.​‌arXiv preprint arXiv:1604.061742016​​back to text
• 67​​​‌ articleD.Daria Cherniuk​, S.Stanislav Abukhovich​‌, A.-H.Anh-Huy Phan​​, I.Ivan Oseledets​​​‌, A.Andrzej Cichocki​ and J.Julia Gusak​‌. Quantization aware factorization​​ for deep neural network​​​‌ compression.Journal of​ Artificial Intelligence Research81​‌2024, 973--988back​​ to text
• 68 article​​​‌R. D.Robert D​ Falgout, S.Stephanie​‌ Friedhoff, T. V.​​Tz V Kolev,​​​‌ S. P.Scott P​ MacLachlan and J. B.​‌Jacob B Schroder.​​ Parallel time integration with​​​‌ multigrid.SIAM Journal​ on Scientific Computing36​‌62014, C635--C661​​back to text
• 69​​​‌ articleM. J.Martin​ J Gander and S.​‌Stefan Vandewalle. Analysis​​ of the parareal time-parallel​​​‌ time-integration method.SIAM​ Journal on Scientific Computing​‌2922007,​​ 556--578back to text​​​‌
• 70 articleP.P.​ Ghysels, X. S.​‌X. S. Li,​​ F.-H.F.-H. Rouet,​​​‌ S.S. Williams and​ A.A. Napov.​‌ An Efficient Multicore Implementation​​ of a Novel HSS-Structured​​​‌ Multifrontal Solver Using Randomized​ Sampling.SIAM Journal​‌ on Scientific Computing38​​52016, S358-S384​​​‌back to text
• 71​ inproceedingsA. N.Aidan​‌ N Gomez, M.​​Mengye Ren, R.​​​‌Raquel Urtasun and R.​ B.Roger B Grosse​‌. The reversible residual​​ network: Backpropagation without storing​​​‌ activations.Proceedings of​ the 31st International Conference​‌ on Neural Information Processing​​ Systems2017, 2211--2221​​​‌back to text
• 72​ inproceedingsA.Audrunas Gruslys​‌, R.Rémi Munos​​, I.Ivo Danihelka​​​‌, M.Marc Lanctot​ and A.Alex Graves​‌. Memory-efficient backpropagation through​​ time.Advances in​​​‌ Neural Information Processing Systems​2016, 4125--4133back​‌ to text
• 73 inproceedings​​U.Udit Gupta,​​​‌ Y. G.Young Geun​ Kim, S.Sylvia​‌ Lee, J.Jordan​​ Tse, H.-H. S.​​​‌Hsien-Hsin S Lee,​ G.-Y.Gu-Yeon Wei,​‌ D.David Brooks and​​ C.-J.Carole-Jean Wu.​​​‌ Chasing Carbon: The Elusive​ Environmental Footprint of Computing​‌.2021 IEEE International​​ Symposium on High-Performance Computer​​​‌ Architecture (HPCA)IEEE2021​, 854--867back to​‌ text
• 74 inproceedingsJ.​​Julia Gusak, D.​​​‌Daria Cherniuk, A.​Alena Shilova, A.​‌Alexander Katrutsa, D.​​Daniel Bershatsky, X.​​​‌Xunyi Zhao, L.​Lionel Eyraud-Dubois, O.​‌Oleg Shlyazhko, D.​​Denis Dimitrov, I.​​Ivan Oseledets and O.​​​‌Olivier Beaumont. Survey‌ on Large Scale Neural‌​‌ Network Training.The​​ 31st International Joint Conference​​​‌ on Artificial Intelligence (IJCAI)‌2022back to text‌​‌back to text
• 75​​ articleA.A. Ida​​​‌, T.T. Iwashita‌, T.T. Mifune‌​‌ and Y.Y. Takahashi​​. Parallel Hierarchical Matrices​​​‌ with Adaptive Cross Approximation‌ on Symmetric Multiprocessing Clusters‌​‌.Journal of Information​​ Processing2242014​​​‌, 642--650back to‌ text
• 76 techreportE.‌​‌Esragul Korkmaz, M.​​Mathieu Faverge, G.​​​‌Grégoire Pichon and P.‌Pierre Ramet. Deciding‌​‌ Non-Compressible Blocks in Sparse​​ Direct Solvers using Incomplete​​​‌ Factorization.RR-9396Inria‌ Bordeaux - Sud Ouest‌​‌2021, 16HAL​​back to text
• 77​​​‌ inproceedingsN.Navjot Kukreja‌, A.Alena Shilova‌​‌, O.Olivier Beaumont​​, J.Jan Huckelheim​​​‌, N.Nicola Ferrier‌, P.Paul Hovland‌​‌ and G.Gerard Gorman​​. Training on the​​​‌ Edge: The why and‌ the how.2019‌​‌ IEEE International Parallel and​​ Distributed Processing Symposium Workshops​​​‌ (IPDPSW)IEEE2019,‌ 899--903back to text‌​‌
• 78 inproceedingsX.Xavier​​ Lacoste, M.Mathieu​​​‌ Faverge, G.George‌ Bosilca, P.Pierre‌​‌ Ramet and S.Samuel​​ Thibault. Taking Advantage​​​‌ of Hybrid Systems for‌ Sparse Direct Solvers via‌​‌ Task-Based Runtimes.2014​​ IEEE International Parallel &​​​‌ Distributed Processing Symposium Workshops,‌ Phoenix, AZ, USA, May‌​‌ 19-23, 2014IEEE Computer​​ Society2014, 29--38​​​‌URL: https://doi.org/10.1109/IPDPSW.2014.9DOIback‌ to text
• 79 book‌​‌T.T. Mary.​​ Block Low-Rank multifrontal solvers:​​​‌ complexity, performance and scalability‌.Université Toulouse 3‌​‌ Paul SabatierPh.D. Dissertation​​2017back to text​​​‌
• 80 articleS.Salli‌ Moustafa, F.François‌​‌ Févotte, M.Mathieu​​ Faverge, L.Laurent​​​‌ Plagne and P.Pierre‌ Ramet. Efficient Parallel‌​‌ Solution of the 3D​​ Stationary Boltzmann Transport Equation​​​‌ for Diffusive Problems.‌Journal of Computational Physics‌​‌March 2019HALDOI​​back to text
• 81​​​‌ inproceedingsD.Deepak Narayanan‌, A.Aaron Harlap‌​‌, A.Amar Phanishayee​​, V.Vivek Seshadri​​​‌, N. R.Nikhil‌ R Devanur, G.‌​‌ R.Gregory R Ganger​​, P. B.Phillip​​​‌ B Gibbons and M.‌Matei Zaharia. PipeDream:‌​‌ generalized pipeline parallelism for​​ DNN training.Proceedings​​​‌ of the 27th ACM‌ Symposium on Operating Systems‌​‌ Principles2019, 1--15​​back to textback​​​‌ to text
• 82 article‌D.David Patterson,‌​‌ J.Joseph Gonzalez,​​ Q.Quoc Le,​​​‌ C.Chen Liang,‌ L.-M.Lluis-Miquel Munguia,‌​‌ D.Daniel Rothchild,​​ D.David So,​​​‌ M.Maud Texier and‌ J.Jeff Dean.‌​‌ Carbon emissions and large​​ neural network training.​​​‌arXiv preprint arXiv:2104.103502021‌back to text
• 83‌​‌ inproceedingsA.-H.Anh-Huy Phan​​, K.Konstantin Sobolev​​​‌, K.Konstantin Sozykin‌, D.Dmitry Ermilov‌​‌, J.Julia Gusak​​, P.Petr Tichavsk\`y​​​‌, V.Valeriy Glukhov‌, I.Ivan Oseledets‌​‌ and A.Andrzej Cichocki​​. Stable Low-rank Tensor​​​‌ Decomposition for Compression of‌ Convolutional Neural Network.‌​‌European Conference on Computer​​​‌ Vision (ECCV)Springer2020​, 522--539back to​‌ textback to text​​
• 84 articleG.Grégoire​​​‌ Pichon, E.Eric​ Darve, M.Mathieu​‌ Faverge, P.Pierre​​ Ramet and J.Jean​​​‌ Roman. Sparse supernodal​ solver using block low-rank​‌ compression: Design, performance and​​ analysis.International Journal​​​‌ of Computational Science and​ Engineering27July 2018​‌, 255 - 270​​HALDOIback to​​​‌ text
• 85 inproceedingsG.​Grégoire Pichon, M.​‌Mathieu Faverge and P.​​Pierre Ramet. Recent​​​‌ Developments Around the Block​ Low-Rank PaStiX Solver.​‌SIAM Conference on Parallel​​ Processing for Scientific Computing​​​‌ (SIAM PP 2020)2020​back to text
• 86​‌ inproceedingsD.Dalal Sukkari​​, H.Hatem Ltaief​​​‌, D.David Keyes​ and M.Mathieu Faverge​‌. Leveraging Task-Based Polar​​ Decomposition Using PARSEC on​​​‌ Massively Parallel Systems.​2019 IEEE International Conference​‌ on Cluster Computing (CLUSTER)​​IEEE2019, 1--12​​​‌back to text