Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.

Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.

For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses. The Commands team itself has dealt since its foundation in 2009 with several types of applications:

We give more details in the Bilateral contracts section.

The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in 38), with improvements due to the “Chicago school”, Bliss 30 during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young 43).

Trajectory optimization really started with the spectacular achievement done by Pontryagin's group 42 during the fifties,
by stating, for general optimal control problems, nonlocal optimality conditions generalizing those of Weierstrass.
This motivated the application to many industrial problems (see the classical books by Bryson and Ho 34, Leitmann 41,
Lee and Markus 40, Ioffe and Tihomirov 37).

Dynamic programming was introduced and systematically studied by R. Bellman during the fifties. The HJB equation, whose solution is the
value function of the (parameterized) optimal control problem, is a variant of the classical Hamilton-Jacobi equation of mechanics
for the case of dynamics parameterized by a control variable. It may be viewed as a differential form of the dynamic programming principle.
This nonlinear first-order PDE appears to be well-posed in the framework of viscosity solutions introduced by Crandall and Lions
35.
The theoretical contributions in this direction did not cease growing, see the books by Barles 29 and Bardi and Capuzzo-Dolcetta 28.

The so-called direct methods consist in an optimization of the trajectory, after having discretized time, by a nonlinear programming solver that possibly
takes into account the dynamic structure. So the two main problems are the choice of the discretization and the nonlinear programming algorithm.
A third problem is the possibility of refinement of the discretization once after solving on a coarser grid.

In the full discretization approach, general Runge-Kutta schemes with different values of control for each inner step are used. This allows to obtain and
control high orders of precision, see Hager 36, Bonnans 31.
In the indirect approach, the control is eliminated thanks to Pontryagin's maximum principle. One has then to solve the two-points boundary value problem
(with differential variables state and costate) by a single or multiple shooting method. The questions are here the choice of a discretization scheme for the
integration of the boundary value problem, of a (possibly globalized) Newton type algorithm for solving the resulting finite dimensional problem
in

This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting when the problem is not convex.

Optimal stochastic control problems occur when the dynamical system is
uncertain. A decision typically has to be taken at each time, while
realizations of future events are unknown (but some information is given
on their distribution of probabilities).
In particular, problems of economic nature deal with large uncertainties
(on prices, production and demand).
Specific examples are the portfolio selection problems in a market with risky
and non-risky assets, super-replication with uncertain volatility,
management of power resources (dams, gas).
Air traffic control is another example of such problems.

For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold 33. Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree 32.

Our current research on aggregative optimization and mean-field games has a natural application to the problem of distributed production of electric energy. We are looking forwaard on developments in that direction.

In collaboration with Ifpen and in the framework of A. Le Rhun's thesis, we have developped a methodology for the optimal energy management for hybrid vehicles, based on a statistical analysis of the traffic. See 17, 17, 39.

In collaboration with the Inbio team (Inst. Pasteur and Inria) we started to study the optimization of protein production based on cell culture.

Our research on large scale aggregative optimization and mean field games have natural applications in the domain of energy production under environmental constraints. So we hope to develop in the next years numerical tools for the reduction of the impact of energy production.

Significant progress has been achieved in the three following research areas:

In 7 we consider an optimal control problem governed by a semilinear heat equation with bilinear control-state terms and subject to control and state constraints. The state constraints are of integral type, the integral being with respect to the space variable. The control is multidimensional. The cost functional is of a tracking type and contains a linear term in the control variables. We derive second order necessary conditions relying on the concept of alternative costates and quasi-radial critical directions. The appendix provides an example illustrating the applicability of our results.

In 8 we continue the previous work by deriving second order sufficient conditions relying on the Goh transform. The appendix provides an example illustrating the applicability of our results.

In the articles 13 and 12, the efficiency of the Receding Horizon method is investigated for two classes of optimal control problems involving PDEs. An explicit rate of convergence is obtained. In the first article, nonlinear stabilization problems are considered. The method takes advantage of computable Taylor expansions of the value function (around some equilibrium point). Linear-quadratic problems are investigated in the second reference. The method takes advantage of the so-called turnpike property, satisfied under a stabilizability and an observability condition. The two works have been written by L. Pfeiffer during his former position at the University of Graz.

In 27, we consider the framework of high dimensional stochastic control problem, in which the controls are aggregated in the cost function. As first contribution we introduce a modified problem, whose optimal control is under some reasonable assumptions an ε-optimal solution of the original problem. As second contribution, we present a decentralized algorithm whose convergence to the solution of the modified problem is established. Finally, we study the application to a problem of coordination of energy consumption and production of domestic appliances.

In 16, optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.

In 26, we consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is in duality with an optimization problem involving the Lagrange multiplier associated with the constraints. Then by convex analysis techniques, we provide a general existence result and some a priori estimation of the dual optimizers. We further provide a necessary and sufficient optimality condition for the initial constrained control problem. The same results are also obtained for a discrete time constrained control problem. Moreover, under additional regularity conditions, it is proved that the discrete time control problem converges to the continuous time problem, possibly with a convergence rate. This convergence result can be used to obtain numerical algorithms to approximate the continuous time control problem, which we illustrate by two simple numerical examples.

In 19 we introduce a new strategy for the design of second-order accurate discretiza-tions of non-linear second order operators of Bellman type, which preserves degenerate ellipticity. The approach relies on Selling's formula, a tool from lattice geometry, and is applied to the Pucci and Monge-Ampere equations, discretized on a two dimensional cartesian grid. In the case of the Monge-Ampere equation, our work is related to both the stable formulation and the second order accurate scheme. Numerical experiments illustrate the robustness and the accuracy of the method.

In 20, we design adaptive finite differences discretizations, which are degenerate elliptic and second order consistent, of linear and quasi-linear partial differential operators featuring both a first order term and an anisotropic second order term. Our approach requires the domain to be discretized on a Cartesian grid, and takes advantage of techniques from the field of low-dimensional lattice geometry. We prove that the stencil of our numerical scheme is optimally compact, in dimension two, and that our approach is quasi-optimal in terms of the compatibility condition required of the first and second order operators, in dimension two and three. Numerical experiments illustrate the efficiency of our method in several contexts.

In 18, using an extension of Varadhan's formula to Randers manifolds, we notice that Randers distances may be approximated by a logarithmic transformation of a linear second-order partial differential equation. Following an idea introduced by Crane, Weischedel, and Wardetzky in the case of Riemannian distances, we study a numerical method for approximating Randers distances which involves a discretization of this linear equation. We propose to use Selling's formula, which originates from the theory of low-dimensional lattice geometry, to build a monotone and linear finite-difference scheme. By injecting the logarithmic transformation in this linear scheme, we are able to prove convergence of this numerical method to the Randers distance, as well as consistency to the order two thirds far from the boundary of the considered domain. We explain how this method may be used to approximate optimal transport distances, how has been previously done in the Riemannian case.

In 21, we propose and investigate a discrete-time mean field game model involving risk-averse agents. The model under study is a coupled system of dynamic programming equations with a Kolmogorov equation. The agents' risk aversion is modeled by composite risk measures. The existence of a solution to the coupled system is obtained with a fixed point approach. The corresponding feedback control allows to construct an approximate Nash equilibrium for a related dynamic game with finitely many players.

In 22, we analyze a system of partial differential equations that model a potential mean field game of controls, briefly MFGC. Such a game describes the interaction of infinitely many negligible players competing to optimize a personal value function that depends in aggregate on the state and, most notably, control choice of all other players. A solution of the system corresponds to a Nash Equilibrium, a group optimal strategy for which no one player can improve by altering only their own action. We investigate the second order, possibly degenerate, case with non-strictly elliptic diffusion operator and local coupling function. The main result exploits potentiality to employ variational techniques to provide a unique weak solution to the system, with additional space and time regularity results under additional assumptions. New analytical subtleties occur in obtaining a priori estimates with the introduction of an additional coupling that depends on the state distribution as well as feedback.

In 15 we discuss discrete-state continuous-time Markov processes, an important class of models employed broadly across the sciences. When the system size becomes large, standard approaches can become intractable to exact solution and numerical simulation. Approximations posed on a continuous state space are often more tractable and are presumed to converge in the limit as the system size tends to infinity. For example, an expansion of the master equation truncated at second order yields the Fokker–Planck equation, a widely used continuum approximation equipped with an underlying process of continuous state. Surprisingly, in [Doering et. al. Multiscale Model. Sim. 2005 3:2, p.283–299] it is shown that the Fokker–Planck approximation may exhibit exponentially large errors, even in the infinite system-size limit. Crucially, the source of this inaccuracy has not been addressed. In this paper, we focus on the family of continuous-state approximations obtained by arbitrary-order truncations. We uncover how the exponentially large error stems from the truncation by quantifying the rapid error decay with increasing truncation order. Furthermore, we explain why this discrepancy only comes to light in a subset of problems. The approximations produced by finite truncation beyond second order lack underlying stochastic processes. Nevertheless, they retain valuable information that explains the previously observed discrepancy by bridging the gap between the continuous and discrete processes. The insight conferred by this broader notion of “continuum approximation”, where we do not require an underlying stochastic process, prompts us to revisit previously expressed doubts regarding continuum approximations. In establishing the utility of higher-order truncations, this approach also contributes to the extensive discussion in the literature regarding the second-order truncation: while recognising the appealing features of an associated stochastic process, in certain cases it may be advantageous to dispense of the process in exchange for the increased approximation accuracy guaranteed by higher-order truncations.

In 25, we discuss the chemical master equation and its continuum approximations, are indispensable tools in the modeling of chemical reaction networks. These are routinely used to capture complex nonlinear phenomena such as multimodality as well as transient events such as first-passage times, that accurately characterise a plethora of biological and chemical processes. However, some mechanisms, such as heterogeneous cellular growth or phenotypic selection at the population level, cannot be represented by the master equation and thus have been tackled separately. In this work, we propose a unifying framework that augments the chemical master equation to capture such auxiliary dynamics, and we develop and analyse a numerical solver that accurately simulates the system dynamics. We showcase these contributions by casting a diverse array of examples from the literature within this framework, and apply the solver to both match and extend previous studies. Analytical calculations performed for each example validate our numerical results and benchmark the solver implementation.

In 10 we study an optimal control problem with state constraints where the state is given by an age-structured, abstract parabolic differential equation. We prove the existence and uniqueness of solution for the state equation and provide first and second parabolic estimates. We analyze the differentiability of the cost function and, based on the general theory of Lagrange multipliers, we give a first order optimality condition. We also define and analyze the regularity of the costate. Finally, we present a pregnancy model, where two coupled age-structured equations are involved, and we apply the obtained results to this case.

In 11, we propose a model for the COVID-19 epidemic where the population is partitioned into classes corresponding to ages (that remain constant during the epidemic). The main feature is to take into account the infection age of the infected population. This allows to better simulate the infection propagation that crucially depend on the infection age. We discuss how to compute the coefficients from data available in the future, and introduce a confinement variable as control. The cost function is a compromise between confinement cost, hospitalization peak and the death toll. Our numerical experiments allow to evaluate the interest of confinement varying with age classes.

In 9, we prove second-order necessary optimality conditions for the so-called time crisis problem that comes up within the context of viability theory. It consists in minimizing the time spent by solutions of a controlled dynamics outside a given subset

Inria Project Lab COSY (started in 2017) aims at exploiting the potential of state-of-art biological modelling, control techniques, synthetic biology and experimental equipment to achieve a paradigm shift in control of microbial communities. More precisely, we plan to determine and implement control strategies to make heterogeneous communities diversify and interact in the most profitable manner. Study of yeast cells has started in collaboration with team Lifeware (G. Batt) in the framework of the PhD of V. Andreani, and is pursued in the Postdoc of D. Lunz (started Nov. 2019).

F. Bonnans is cofounder and Associate Editor of Math. and Applications, Annals of AOSR (Academy of Science of Romania).

Reviews for major journals in the field.

F. Bonnans is member of the PGMO (Program Gaspard Monge in Optimization) board.

2nd year of master / 3rd year of engineering school:

L. Pfeiffer is correspondent for the hiring mission (mission recrutement).