Section: Research Program
Participants : Laurent Hascoët, Valérie Pascual, Ala Taftaf.
- algorithmic differentiation
(AD, aka Automatic Differentiation) Transformation of a program, that returns a new program that computes derivatives of the initial program, i.e. some combination of the partial derivatives of the program's outputs with respect to its inputs.
Algorithmic Differentiation (AD) differentiates programs. The input of AD is a source program that, given some , returns some , for a differentiable . AD generates a new source program that, given , computes some derivatives of  .
The resulting reuses the control of . For any given control, is equivalent to a sequence of instructions, which is identified with a composition of vector functions. Thus, if
where each is the elementary function implemented by instruction . AD applies the chain rule to obtain derivatives of . Calling the values of all variables after instruction , i.e. and , the Jacobian of is
which can be mechanically written as a sequence of instructions . Combining the with the control of yields , and therefore this differentiation is piecewise.
AD can be generalized to higher level derivatives, Taylor series, etc. In practice, many applications only need cheaper projections of such as:
This expression is most efficiently computed from right to left, because matrixvector products are cheaper than matrixmatrix products. This defines the adjoint mode of AD, most effective for optimization, data assimilation  , adjoint problems  , or inverse problems.
Adjoint-mode AD turns out to make a very efficient program, at least theoretically  . The computation time required for the gradient is only a small multiple of the run-time of . It is independent from the number of parameters . In contrast, computing the same gradient with the tangent mode would require running the tangent differentiated program times.
However, the are required in the inverse of their computation order. If the original program overwrites a part of , the differentiated program must restore before it is used by . Therefore, the central research problem of adjoint-mode AD is to make the available in reverse order at the cheapest cost, using strategies that combine storage, repeated forward computation from available previous values, or even inverted computation from available later values.
Another research issue is to make the AD model cope with the constant evolution of modern language constructs. From the old days of Fortran77, novelties include pointers and dynamic allocation, modularity, structured data types, objects, vectorial notation and parallel communication. We keep developing our models and tools to handle these new constructs.