Section: New Results

Differentiation of non-smooth programs

Participants : Laurent Hascoët, Sri Hari Krishna Narayanan [Argonne National Lab. (Illinois, USA)] .

Algorithmic Differentiation can be used to derive tangent models that cope with a certain class of non-smoothness, through the use of the so-called Abs-Normal Form (ANF)  [23]. These tangent models incorporate some knowledge of the nearby discontinuities of the derivatives. These models bring some additional power to processes that use tangent approximations, such as simulation, optimization, or solution of differential equations.

The mechanics to derive these special tangent models can be built as an extension of standard tangent linear Algorithmic Differentiation. This has been first demonstrated by the AD tool AdolC which, being based on Operator Overloading, is more flexible and seems a natural choice for implementation. Together with Krishna Narayanan, we recently tried a similar adaption on Source-Transformation AD tools. It appears that very little development is needed in the AD-tool. Specifically for Tapenade, it appears that no development at all is needed in the tool itself. Any end-user can already produce ANF tangent without needing any access to the tool source. All it requires is a customized derivative of the absolute-value function (ABS), which is currently less than 40 lines of code.

Building the ANF of a given program introduces one new variable per run-time execution of the ABS function. As the number of rows and columns of the constructed extended Jacobian both grow like the number of variables, it may become unreasonably large for large codes. To overcome this issue, we explore the possibility of finding at run-time the "important" ABS calls that deserve this treatment, and those that don't. We base this decision on a notion of distance to the kink induced by this ABS call as illustrated by Figure 2. We presented these experiments at a Shonan meeting on this question (Shonan, Japan, June 25-29) and at a workshop of ISMP 2018 (Bordeaux, July 2-6)

Figure 2. Abs-Normal Form of a non-smooth function: purple:original function, green ANF computed at (30,30). The ANF is linearized around the point of interest, and at the same time captures the non-smooth behavior. Notice the ANF divergence from the original function on the left, due to neglecting the leftmost kink which was decided “far” enough from the point of interest. The ANF divergence on the right is the natural effect of linearization