2.1 Scientific directions

Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.

Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.

2.2 Industrial impact

For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses. The Commands team itself has dealt since its foundation in 2009 with several types of applications:

• Space vehicle trajectories, in collaboration with CNES, the French space agency.
• Aeronautics, in collaboration with the startup Safety Line.
• Production, management, storage and trading of energy resources, in collaboration with Edf, ex-Gdf and Total.
• Energy management for hybrid vehicles, in collaboration with Renault and Ifpen.

We give more details in the Bilateral contracts section.

3.1 Historical aspects

The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in 38), with improvements due to the “Chicago school”, Bliss 30 during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young 43).

Trajectory optimization really started with the spectacular achievement done by Pontryagin's group 42 during the fifties, by stating, for general optimal control problems, nonlocal optimality conditions generalizing those of Weierstrass. This motivated the application to many industrial problems (see the classical books by Bryson and Ho 34, Leitmann 41, Lee and Markus 40, Ioffe and Tihomirov 37).

Dynamic programming was introduced and systematically studied by R. Bellman during the fifties. The HJB equation, whose solution is the value function of the (parameterized) optimal control problem, is a variant of the classical Hamilton-Jacobi equation of mechanics for the case of dynamics parameterized by a control variable. It may be viewed as a differential form of the dynamic programming principle. This nonlinear first-order PDE appears to be well-posed in the framework of viscosity solutions introduced by Crandall and Lions 35. The theoretical contributions in this direction did not cease growing, see the books by Barles 29 and Bardi and Capuzzo-Dolcetta 28.

3.2 Trajectory optimization

The so-called direct methods consist in an optimization of the trajectory, after having discretized time, by a nonlinear programming solver that possibly takes into account the dynamic structure. So the two main problems are the choice of the discretization and the nonlinear programming algorithm. A third problem is the possibility of refinement of the discretization once after solving on a coarser grid.

In the full discretization approach, general Runge-Kutta schemes with different values of control for each inner step are used. This allows to obtain and control high orders of precision, see Hager 36, Bonnans 31. In the indirect approach, the control is eliminated thanks to Pontryagin's maximum principle. One has then to solve the two-points boundary value problem (with differential variables state and costate) by a single or multiple shooting method. The questions are here the choice of a discretization scheme for the integration of the boundary value problem, of a (possibly globalized) Newton type algorithm for solving the resulting finite dimensional problem in ${ℝ}^n$ ($n$ is the number of state variables), and a methodology for finding an initial point.

3.3 Hamilton-Jacobi-Bellman approach

This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting when the problem is not convex.

Optimal stochastic control problems occur when the dynamical system is uncertain. A decision typically has to be taken at each time, while realizations of future events are unknown (but some information is given on their distribution of probabilities). In particular, problems of economic nature deal with large uncertainties (on prices, production and demand). Specific examples are the portfolio selection problems in a market with risky and non-risky assets, super-replication with uncertain volatility, management of power resources (dams, gas). Air traffic control is another example of such problems.

For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold 33. Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree 32.

4.1 Electric energy production

Our current research on aggregative optimization and mean-field games has a natural application to the problem of distributed production of electric energy. We are looking forwaard on developments in that direction.

4.2 Energy management for hybrid vehicles

In collaboration with Ifpen and in the framework of A. Le Rhun's thesis, we have developped a methodology for the optimal energy management for hybrid vehicles, based on a statistical analysis of the traffic. See 17, 17, 39.

4.3 Biological cells culture

In collaboration with the Inbio team (Inst. Pasteur and Inria) we started to study the optimization of protein production based on cell culture.

5.1 Impact of research results

Our research on large scale aggregative optimization and mean field games have natural applications in the domain of energy production under environmental constraints. So we hope to develop in the next years numerical tools for the reduction of the impact of energy production.

7.1 Optimal control of partial differential equations

7.2 Stochastic control and HJB equations

In 27, we consider the framework of high dimensional stochastic control problem, in which the controls are aggregated in the cost function. As first contribution we introduce a modified problem, whose optimal control is under some reasonable assumptions an ε-optimal solution of the original problem. As second contribution, we present a decentralized algorithm whose convergence to the solution of the modified problem is established. Finally, we study the application to a problem of coordination of energy consumption and production of domestic appliances.

In 16, optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

In 26, we consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is in duality with an optimization problem involving the Lagrange multiplier associated with the constraints. Then by convex analysis techniques, we provide a general existence result and some a priori estimation of the dual optimizers. We further provide a necessary and sufficient optimality condition for the initial constrained control problem. The same results are also obtained for a discrete time constrained control problem. Moreover, under additional regularity conditions, it is proved that the discrete time control problem converges to the continuous time problem, possibly with a convergence rate. This convergence result can be used to obtain numerical algorithms to approximate the continuous time control problem, which we illustrate by two simple numerical examples.

7.3 Numerical analysis of variational problems

In 19 we introduce a new strategy for the design of second-order accurate discretiza-tions of non-linear second order operators of Bellman type, which preserves degenerate ellipticity. The approach relies on Selling's formula, a tool from lattice geometry, and is applied to the Pucci and Monge-Ampere equations, discretized on a two dimensional cartesian grid. In the case of the Monge-Ampere equation, our work is related to both the stable formulation and the second order accurate scheme. Numerical experiments illustrate the robustness and the accuracy of the method.

In 20, we design adaptive finite differences discretizations, which are degenerate elliptic and second order consistent, of linear and quasi-linear partial differential operators featuring both a first order term and an anisotropic second order term. Our approach requires the domain to be discretized on a Cartesian grid, and takes advantage of techniques from the field of low-dimensional lattice geometry. We prove that the stencil of our numerical scheme is optimally compact, in dimension two, and that our approach is quasi-optimal in terms of the compatibility condition required of the first and second order operators, in dimension two and three. Numerical experiments illustrate the efficiency of our method in several contexts.

In 18, using an extension of Varadhan's formula to Randers manifolds, we notice that Randers distances may be approximated by a logarithmic transformation of a linear second-order partial differential equation. Following an idea introduced by Crane, Weischedel, and Wardetzky in the case of Riemannian distances, we study a numerical method for approximating Randers distances which involves a discretization of this linear equation. We propose to use Selling's formula, which originates from the theory of low-dimensional lattice geometry, to build a monotone and linear finite-difference scheme. By injecting the logarithmic transformation in this linear scheme, we are able to prove convergence of this numerical method to the Randers distance, as well as consistency to the order two thirds far from the boundary of the considered domain. We explain how this method may be used to approximate optimal transport distances, how has been previously done in the Riemannian case.

7.4 Mean-field games

In 21, we propose and investigate a discrete-time mean field game model involving risk-averse agents. The model under study is a coupled system of dynamic programming equations with a Kolmogorov equation. The agents' risk aversion is modeled by composite risk measures. The existence of a solution to the coupled system is obtained with a fixed point approach. The corresponding feedback control allows to construct an approximate Nash equilibrium for a related dynamic game with finitely many players.

In 22, we analyze a system of partial differential equations that model a potential mean field game of controls, briefly MFGC. Such a game describes the interaction of infinitely many negligible players competing to optimize a personal value function that depends in aggregate on the state and, most notably, control choice of all other players. A solution of the system corresponds to a Nash Equilibrium, a group optimal strategy for which no one player can improve by altering only their own action. We investigate the second order, possibly degenerate, case with non-strictly elliptic diffusion operator and local coupling function. The main result exploits potentiality to employ variational techniques to provide a unique weak solution to the system, with additional space and time regularity results under additional assumptions. New analytical subtleties occur in obtaining a priori estimates with the introduction of an additional coupling that depends on the state distribution as well as feedback.

7.5 Biology, medecine

In 15 we discuss discrete-state continuous-time Markov processes, an important class of models employed broadly across the sciences. When the system size becomes large, standard approaches can become intractable to exact solution and numerical simulation. Approximations posed on a continuous state space are often more tractable and are presumed to converge in the limit as the system size tends to infinity. For example, an expansion of the master equation truncated at second order yields the Fokker–Planck equation, a widely used continuum approximation equipped with an underlying process of continuous state. Surprisingly, in [Doering et. al. Multiscale Model. Sim. 2005 3:2, p.283–299] it is shown that the Fokker–Planck approximation may exhibit exponentially large errors, even in the infinite system-size limit. Crucially, the source of this inaccuracy has not been addressed. In this paper, we focus on the family of continuous-state approximations obtained by arbitrary-order truncations. We uncover how the exponentially large error stems from the truncation by quantifying the rapid error decay with increasing truncation order. Furthermore, we explain why this discrepancy only comes to light in a subset of problems. The approximations produced by finite truncation beyond second order lack underlying stochastic processes. Nevertheless, they retain valuable information that explains the previously observed discrepancy by bridging the gap between the continuous and discrete processes. The insight conferred by this broader notion of “continuum approximation”, where we do not require an underlying stochastic process, prompts us to revisit previously expressed doubts regarding continuum approximations. In establishing the utility of higher-order truncations, this approach also contributes to the extensive discussion in the literature regarding the second-order truncation: while recognising the appealing features of an associated stochastic process, in certain cases it may be advantageous to dispense of the process in exchange for the increased approximation accuracy guaranteed by higher-order truncations.

In 25, we discuss the chemical master equation and its continuum approximations, are indispensable tools in the modeling of chemical reaction networks. These are routinely used to capture complex nonlinear phenomena such as multimodality as well as transient events such as first-passage times, that accurately characterise a plethora of biological and chemical processes. However, some mechanisms, such as heterogeneous cellular growth or phenotypic selection at the population level, cannot be represented by the master equation and thus have been tackled separately. In this work, we propose a unifying framework that augments the chemical master equation to capture such auxiliary dynamics, and we develop and analyse a numerical solver that accurately simulates the system dynamics. We showcase these contributions by casting a diverse array of examples from the literature within this framework, and apply the solver to both match and extend previous studies. Analytical calculations performed for each example validate our numerical results and benchmark the solver implementation.

7.6 Other problems

In 9, we prove second-order necessary optimality conditions for the so-called time crisis problem that comes up within the context of viability theory. It consists in minimizing the time spent by solutions of a controlled dynamics outside a given subset $K$ of the state space. One essential feature is the discontinuity of the characteristic function involved in the cost functional. Thanks to a change of time and an augmentation of the dynamics, we relate the time crisis problem to an auxiliary Mayer control problem. This allows us to use the classical tools of optimal control for obtaining optimality conditions. Going back to the original problem, we deduce that way second order optimality conditions for the time crisis problem.

8.1 National initiatives

9.1 Promoting scientific activities

9.2 Teaching - Supervision - Juries

9.3 Popularization

10.1 Major publications

• 1 article Joseph FrédéricJ. Bonnans and JustinaJ. Gianatti. Optimal control techniques based on infection age for the study of the COVID-19 epidemic Mathematical Modelling of Natural Phenomena 2021
  HAL
• 2 articleJoseph FrédéricJ. Bonnans, SaeedS. Hadikhanloo and LaurentL. Pfeiffer. Schauder Estimates for a Class of Potential Mean Field Games of ControlsApplied Mathematics and OptimizationJuly 2019, 34
  HAL
• 3 article ArthurA. Le Rhun, FrédéricF. Bonnans, GiovanniG. De Nunzio, ThomasT. Leroy and PierreP. Martinon. A stochastic data-based traffic model applied to vehicles energy consumption estimation IEEE Transactions on Intelligent Transportation Systems 2019
  HAL DOI
• 4 articleLaurentL. Pfeiffer. Optimality conditions in variational form for non-linear constrained stochastic control problemsMathematical Control & Related Fields1032020, 493-526
  HALDOI
• 5 articleCédricC. Rommel, FrédéricF. Bonnans, PierreP. Martinon and BaptisteB. Gregorutti. Gaussian Mixture Penalty for Trajectory Optimization ProblemsJournal of Guidance, Control, and Dynamics428August 2019, 1857--1862
  HALDOI
• 6 inproceedings EliseE. Weill, VirgileV. Andreani, ChetanC. Aditya, PierreP. Martinon, JakobJ. Ruess, GrégoryG. Batt and FrédéricF. Bonnans. Optimal control of an artificial microbial differentiation system for protein bioproductioń ECC 2019 - European Control Conference Naples, Italy June 2019
  HAL

10.2 Publications of the year

10.3 Cited publications

• 28 book M. Bardi and I. Capuzzo-Dolcetta. Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations Systems and Control: Foundations and Applications Birkhäuser, Boston 1997
  back to text
• 29 book G. Barles. Solutions de viscosité des équations de Hamilton-Jacobi 17 Mathématiques et Applications Springer, Paris 1994
  back to text
• 30 book G.A.G. Bliss. Lectures on the Calculus of Variations Chicago, Illinois University of Chicago Press 1946
  back to text
• 31 articleJoseph FrédéricJ. Bonnans and JulienJ. Laurent-Varin. Computation of order conditions for symplectic partitioned Runge-Kutta schemes with application to optimal controlNumerische Mathematik10312006, 1--10
  back to text
• 32 articleJoseph FrédéricJ. Bonnans, E. Ottenwaelter and HasnaaH. Zidani. Numerical schemes for the two dimensional second-order HJB equationESAIM: M2AN382004, 723-735
  back to text
• 33 articleJoseph FrédéricJ. Bonnans and HasnaaH. Zidani. Consistency of generalized finite difference schemes for the stochastic HJB equationSIAM J. Numerical Analysis412003, 1008-1021
  back to text
• 34 book A. E.A. Bryson and Y.-C. Ho. Applied optimal control New-York Hemisphere Publishing 1975
  back to text
• 35 articleM.G.M. Crandall and P.L.P. Lions. Viscosity solutions of Hamilton Jacobi equationsBull. American Mathematical Society2771983, 1--42
  back to text
• 36 articleW.W.W. Hager. Runge-Kutta methods in optimal control and the transformed adjoint systemNumerische Mathematik8722000, 247--282
  back to text
• 37 book A.D.A. Ioffe and V.M.V. Tihomirov. Theory of Extremal Problems Russian Edition: Nauka, Moscow, 1974 North-Holland Publishing Company, Amsterdam 1979
  back to text
• 38 book J.-L. Lagrange. Mécanique analytique reprinted by J. Gabay, 1989 Paris 1788
  back to text
• 39 articleArthurA. Le Rhun, FrédéricF. Bonnans, GiovanniG. De Nunzio, ThomasT. Leroy and PierreP. Martinon. A stochastic data-based traffic model applied to vehicles energy consumption estimationIEEE Transactions on Intelligent Transportation Systems2172019, 3025--3034
  back to text
• 40 book E.B.E. Lee and L. Markus. Foundations of optimal control theory John Wiley, New York 1967
  back to text
• 41 book G. Leitmann. An introduction to optimal control Mc Graw Hill, New York 1966
  back to text
• 42 book L. Pontryagin, V. Boltyanski, R. Gamkrelidze and E. Michtchenko. The Mathematical Theory of Optimal Processes Wiley Interscience, New York 1962
  back to text
• 43 book L.C.L. Young. Lectures on the calculus of variations and optimal control theory Philadelphia W. B. Saunders Co. 1969
  back to text