Section: New Results

Automatic Differentiation and parallel codes

Participants : Valérie Pascual, Laurent Hascoët, Hubert Alcin, Jean Utke [Argonne National Lab. (Illinois, USA)] , Uwe Naumann [RWTH Aachen University (Germany)] .

Together with colleagues in Argonne National Lab. and RWTH Aachen, we are studying how AD tools can handle MPI-parallel codes, especially in adjoint mode.

This year, we have presented our strategy [16] to extend Data-Flow analysis to Message-Passing communication. This strategy is specially designed for a program representation like that of tapenade , i.e. based on a Call-Graph whose nodes are indeed Flow-Graphs. This representation makes it easier to implement analyses in a way that is both context-sensitive and flow-sensitive. Our strategy also relies on the fixed-point implementation of the analyses, which uses a “wait-list”.

At the same time, we continue the design of a adjoint-mode AD adapted to MPI communication. In our framework of AD by source transformation, we have pushed far in the direction of static data-flow analyses and static source transformation of individual MPI calls. We obtained results on classical cases of message-passing  [38] . However, experience shows [11] that general usage of message-passing defies static analysis. A purely static analysis and transformation must resort too often to conservative choices, yielding a poor efficiency.

As a consequence, we are now going in the direction of a more dynamic, run-time treatment of adjoint MPI calls. This means designing a wrapper library “AMPI” on top of MPI, that takes care during execution of the adjoint code of the bookkeeping to send the adjoint messages in the reverse direction. This wrapper library should also be independent from the particular AD tool, as it will be used not only with tapenade but with the tools developed at Argonne and RWTH Aachen.