Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.
Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.
For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses. The Commands team itself has dealt since its foundation in 2009 with several types of applications:
Space vehicle trajectories, in collaboration with CNES, the French space agency.
Aeronautics, in collaboration with the startup Safety Line.
Production, management, storage and trading of energy resources, in collaboration with EDF, GDF and TOTAL.
Energy management for hybrid vehicles, in collaboration with Renault and IFPEN.
We give more details in the Bilateral contracts section.
The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in ), with improvements due to the “Chicago school”, Bliss during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young ).
Trajectory optimization really started with the spectacular achievement done by Pontryagin's group during the fifties, by stating, for general optimal control problems, nonlocal optimality conditions generalizing those of Weierstrass. This motivated the application to many industrial problems (see the classical books by Bryson and Ho , Leitmann , Lee and Markus , Ioffe and Tihomirov ).
Dynamic programming was introduced and systematically studied by R. Bellman during the fifties. The HJB equation, whose solution is the value function of the (parameterized) optimal control problem, is a variant of the classical Hamilton-Jacobi equation of mechanics for the case of dynamics parameterized by a control variable. It may be viewed as a differential form of the dynamic programming principle. This nonlinear first-order PDE appears to be well-posed in the framework of viscosity solutions introduced by Crandall and Lions . The theoretical contributions in this direction did not cease growing, see the books by Barles and Bardi and Capuzzo-Dolcetta .
The so-called direct methods consist in an optimization of the trajectory, after having discretized time, by a nonlinear programming solver that possibly takes into account the dynamic structure. So the two main problems are the choice of the discretization and the nonlinear programming algorithm. A third problem is the possibility of refinement of the discretization once after solving on a coarser grid.
In the full discretization approach, general Runge-Kutta schemes with different values of control for each inner step are used. This allows to obtain and
control high orders of precision, see Hager , Bonnans .
In the indirect approach, the control is eliminated thanks to Pontryagin's maximum principle. One has then to solve the two-points boundary value problem
(with differential variables state and costate) by a single or multiple shooting method. The questions are here the choice of a discretization scheme for the
integration of the boundary value problem, of a (possibly globalized) Newton type algorithm for solving the resulting finite dimensional problem
in
This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting when the problem is not convex.
Optimal stochastic control problems occur when the dynamical system is uncertain. A decision typically has to be taken at each time, while realizations of future events are unknown (but some information is given on their distribution of probabilities). In particular, problems of economic nature deal with large uncertainties (on prices, production and demand). Specific examples are the portfolio selection problems in a market with risky and non-risky assets, super-replication with uncertain volatility, management of power resources (dams, gas). Air traffic control is another example of such problems.
For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold . Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree .
We have a collaboration with the startup Safety Line on the optimization of trajectories for civil aircrafts. Key points include the reliable identification of the plane parameters (aerodynamic and thrust models) using data from the flight recorders, and the robust trajectory optimization of the climbing and cruise phases. We use both local (quasi-Newton interior-point algorithms) and global optimization tools (dynamic programming). The local method for the climb phase is in production and has been used for several hundreds of actual plane flights.
We have a collaboration with IFPEN on the energy management for hybrid vehicles. A significant direction is the analysis and classification of traffic data. More specifically, we focus on the traffic probability distribution in the (speed,torque) plane, with a time / space subdivision (road segments and timeframes).
We renewed in 2017 our interest in (micro)biological systems, joining projects Cosy and Algae in silico on the topic of the optimization of micro-organisms populations.
In , J. Garcke (SCAI-Fraunhofer I.) and A. Kröner were able to solve finite time horizon suboptimal feedback control problems for partial differential equations is proposed by solving dynamic programming equations on adaptive sparse grids. The approach is illustrated for the wave equation and an extension to equations of Schrödinger type is discussed. A semi-discrete optimal control problem is introduced and the feedback control is derived from the corresponding value function. A semi-Lagrangian scheme is combined with spatially adaptive sparse grids. An adaptive grid refinement procedure is explored. We present several numerical examples studying the effect the parameters characterizing the sparse grid have on the accuracy of the value function and the optimal trajectory. Problems with dimensions up to eight were solved.
Boite à Outils pour le Contrôle OPtimal
Keywords: Dynamic Optimization - Identification - Biology - Numerical optimization - Energy management - Transportation
Functional Description: Bocop is an open-source toolbox for solving optimal control problems, with collaborations with industrial and academic partners. Optimal control (optimization of dynamical systems governed by differential equations) has numerous applications in transportation, energy, process optimization, energy and biology. Bocop includes a module for parameter identification and a graphical interface, and runs under Linux / Windows / Mac.
Release Functional Description: Handling of delay systems Alternate automatic differentiation tool: CppAD Update for CMake and MinGW (windows version)
Participants: Benjamin Heymann, Virgile Andreani, Jinyan Liu, Joseph Frédéric Bonnans and Pierre Martinon
Contact: Pierre Martinon
URL: http://
Keywords: Optimal control - Stochastic optimization - Global optimization
Functional Description: Toolbox for stochastic or deterministic optimal control, dynamic programming / HJB approach.
Release Functional Description: User interface State jumps for switched systems Explicit handling of final conditions Computation of state probability density (fiste step to mean field games)
Participants: Benjamin Heymann, Jinyan Liu, Joseph Frédéric Bonnans and Pierre Martinon
Contact: Joseph Frédéric Bonnans
URL: http://
Keywords: Optimization - Aeronautics
Functional Description: Optimize the climb speeds and associated fuel consumption for the flight planning of civil airplanes.
News Of The Year: Improved atmosphere model 2D interpolations for temperature and wind data
Participants: Gregorutti Baptiste, Cindie Andrieu, Anamaria Lupu, Joseph Frédéric Bonnans, Karim Tekkal, Pierre Jouniaux and Pierre Martinon
Partner: Safety Line
Contact: Pierre Martinon
Keywords: Optimization - Aeronautics
Functional Description: Optimize the climb and cruising trajectory of flight by a HJB approach.
News Of The Year: First demonstrator for cruise flight deployed at Safety Line
Participants: Pierre Martinon, Joseph Frédéric Bonnans, Jinyan Liu, Gregorutti Baptiste and Anamaria Lupu
Partner: Safety Line
Contact: Pierre Martinon
With Mickaël D. Chekroun (UCLA), and Honghu Liu (Virginia Tech).
Nonlinear optimal control problems in Hilbert spaces are considered
for which we derive approximation theorems for Galerkin
approximations. Approximation theorems are available in the
literature. The originality of our approach relies on the
identification of a set of natural assumptions that allows us to deal
with a broad class of nonlinear evolution equations and cost
functionals for which we derive convergence of the value functions
associated with the optimal control problem of the Galerkin
approximations. This convergence result holds for a broad class of
nonlinear control strategies as well. In particular, we show that the
framework applies to the optimal control of semilinear heat equations
posed on a general compact manifold without boundary. The framework is
then shown to apply to geoengineering and mitigation of greenhouse gas
emissions formulated for the first time in terms of optimal control of
energy balance climate models posed on the sphere
With Mickaël D. Chekroun (UCLA), and Honghu Liu (Virginia Tech).
Optimal control problems of nonlinear delay differential equations (DDEs) are considered for which we propose a general Galerkin approximation scheme built from Koornwinder polynomials. Error estimates for the resulting Galerkin-Koornwinder approximations to the optimal control and the value function, are derived for a broad class of cost function-als and nonlinear DDEs. The approach is illustrated on a delayed logistic equation set not far away from its Hopf bifurcation point in the parameter space. In this case, we show that low-dimensional controls for a standard quadratic cost functional can be efficiently computed from Galerkin-Koornwinder approximations to reduce at a nearly optimal cost the oscillation amplitude displayed by the DDE's solution. Optimal controls computed from the Pontryagin's maximum principle (PMP) and the Hamilton-Jacobi-Bellman equation (HJB) associated with the corresponding ODE systems, are shown to provide numerical solutions in good agreement. It is finally argued that the value function computed from the corresponding reduced HJB equation provides a good approximation of that obtained from the full HJB equation. See .
With Justina Gianatti (U. Rosario) and Francisco J. Silva (U. Limoges) In this work we consider the time discretization of stochastic optimal control problems. Under general assumptions on the data, we prove the convergence of the value functions associated with the discrete time problems to the value function of the original problem. Moreover, we prove that any sequence of optimal solutions of discrete problems is minimizing for the continuous one. As a consequence of the Dynamic Programming Principle for the discrete problems, the minimizing sequence can be taken in discrete time feedback form. See .
We perform a variational analysis for a class of European or American options with stochastic volatility models, including those of Heston and Achdou-Tchou. Taking into account partial correlations and the presence of multiple factors, we obtain the well-posedness of the related partial differential equations, in some weighted Sobolev spaces. This involves a generalization of the commutator analysis introduced by Achdou and Tchou. See .
With Athena Picarelli (U. Oxford) and Hasna Zidani (ENSTA).
An infinite horizon stochastic optimal control problem with running maximum cost is considered. The value function is characterized as the viscosity solution of a second-order HJB equation with mixed boundary condition. A general numerical scheme is proposed and convergence is established under the assumptions of consistency, monotonicity and stability of the scheme. A convergent semi-Lagrangian scheme is presented in detail. See .
With Pierre Picard and Anasuya Raj (Ecole Polytechnique, Econ. dpt).
We analyze in the design of optimal medical insurance under ex post moral hazard, i.e., when illness severity cannot be observed by insurers and policyholders decide on their health expenditures. We characterize the trade-off between ex ante risk sharing and ex post incentive compatibility, in an optimal revelation mechanism under hidden information and risk aversion.
We establish that the optimal contract provides partial insurance at the margin, with a deductible when insurers rates are affected by a positive loading, and that it may also include an upper limit on coverage. We show that the potential to audit the health state leads to an upper limit on out-of-pocket expenses. Numerical simulations indicate that these qualitative results tend to be robust with respect to the health parameter.
In the framework of an Ilab with startup Safety Line (http://
A second part is optimizing the fuel consumption during the climb and cruise phases. Fig. shows a simulated climb phase, along with recorded data from the actual flight. This collaboration relies significantly on the toolboxes Bocop and BocopHJB developed by Commands since 2010. The resulting commercial tool OptiClimb is currently under testing in several airplane companies, totalling about a hundred actual optimized flights per day. Recent improvements include better atmosphere models and more accurate data for temperature and wind, as well as a first demonstrator for cruise flight optimization, see Fig. .
This study is presently conducted in the framework of the PhD of Arthur Le Rhun, started in Fall 2016. The main axis is to design a traffic model suitable for optimizing the fuel consumption of a hybrid vehicle following a given route. The first step was to develop a new traffic model in which the consumption is infered only on the functionning points in the (speed,torque) plane. More precisely, we are interested in the probability distribution of these functionning points when considering a space/time subdivision into road segments and timeframes (see Fig.). In order to reduce the huge number of distributions obtained, we perform a clustering step using k-means (Fig.). Since the objects to be clustered are distributions, we choose to use the Wasserstein distance based on optimal transport. The task of computing these Wasserstein barycenters was done by Sinkhorn iterations, and we also developped a variant of stochastic gradient that scales better for huge data sets.
In order to obtain the data for our traffic analysis, we work with
a traffic simulator called SUMO, with the LUST scenario modeling the city of Luxembourg (http://
Gaspard Monge Program for Optimization and Operational Research (Fondation Jacques Hadamard)
Inria Project Lab COSY (started in 2017) aims at exploiting the potential of state-of-art biological modelling, control techniques, synthetic biology and experimental equipment to achieve a paradigm shift in control of microbial communities. More precisely, we plan to determine and implement control strategies to make heterogeneous communities diversify and interact in the most profitable manner. Study of yeast cells has started in collaboration with team Lifeware (G. Batt) in the framework of the PhD of V. Andreani.
Inria Project Lab ALGAE IN SILICO (started in 2014) is dedicated to provide an integrated platform for numerical simulation of microalgae “from genes to industrial process“. The project has now reached a stage where we can tackle the optimization aspects. Commands is currently joining the IPL, in the following of our previous collaborations with teams Modemic and Biocore on bioreactors, see ,
Joao Miguel Machado, from FGV (Rio de Janeiro), spent his master internship in our team from sept-dec 2017, working with F. Bonnans and M.S. Aronna (EMAP-FGV) on the second order necessary and sufficient optimality conditions for optimal control problems of ODEs with broken extremals, i.e., with discontinuous control. We are currently extending the classical theory to the case of a jump between interior and boundary values for the control.
F. Bonnans: PGMO Days 2017.
F. Bonnans: Associate Editor of “Applied Mathematics and Optimization” and of “Series on Mathematics and its Applications, Annals of The Academy of Romanian Scientists”.
Reviews in 2017 for major journals in the field: Applied Mathematics and Optimization, Automatica, Int. J. of Control, Inverse problems, J. Convex Analysis, J. Diff. Equations, J. of Optimization Theory and Applications, Optimization Set Valued and Variational Analysis, SIAM J. Optimization, SIAM J. Control and Optimization, several conference proceedings.
F. Bonnans: Forecasting and risk management for renewable energy, June 7-9, Paris; Numoc, June 19-23, Roma; NHOC2017, July 3-5, Porto; Optimal Control of Partial Differential Equations, Sept, Castro Urdiales.
F. Bonnans: French representative to the IFIP-TC7 committee (International Federation of Information Processing; TC7 devoted to System Modeling and Optimization).
F. Bonnans: member of the PGMO board and Steering Committee (Gaspard Monge Program for Optimization and Operations Research, EDF-FMJH).
Master :
F. Bonnans: Numerical analysis of partial differential equations arising in finance and stochastic control, 24h, M2, Ecole Polytechnique and U. Paris 6, France.
F. Bonnans: Optimal control, 15h, M2, Optimization master (U. Paris-Saclay) and Ensta, France.
F. Bonnans: Stochastic optimization, 15h, M2, Optimization master (U. Paris-Saclay), France.
A. Kröner : Optimal control of partial differential equations, 20h, M2, Optimization master (U. Paris-Saclay), France.
PhD in progress : Cédric Rommel, Data exploration for the optimization of aircraft trajectories. Started November 2015 (CIFRE fellowship with Safety Line), F. Bonnans and P. Martinon.
PhD in progress : Arthur Le Rhun, Optimal and robust control of hybrid vehicles. Started September 2016 (IFPEN fellowship), F. Bonnans and P. Martinon.
The collaboration with startup Safety Line was presented at events “Vivatech” (17/06/2017, https://