Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.
Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.
For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses or of postdocs. The Commands team itself has dealt since its foundation in 2007 with several types of applications:
Space vehicle trajectories, in collaboration with CNES, the French space agency.
Aeronautics, in collaboration with the startup Safety Line.
Production, management, storage and trading of energy resources, in collaboration with EDF, GDF and TOTAL.
Energy management for hybrid vehicles, in collaboration with Renault and IFPEN.
We give more details in the Bilateral contracts section.
The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in ), with improvements due to the “Chicago school”, Bliss during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young ).
Trajectory optimization really started with the spectacular achievement done by Pontryagin's group during the fifties, by stating, for general optimal control problems, nonlocal optimality conditions generalizing those of Weierstrass. This motivated the application to many industrial problems (see the classical books by Bryson and Ho , Leitmann , Lee and Markus , Ioffe and Tihomirov ). Since then, various theoretical achievements have been obtained by extending the results to nonsmooth problems, see Aubin , Clarke , Ekeland .
Dynamic programming was introduced and systematically studied by R. Bellman during the fifties. The HJB equation, whose solution is the value function of the (parameterized) optimal control problem, is a variant of the classical Hamilton-Jacobi equation of mechanics for the case of dynamics parameterized by a control variable. It may be viewed as a differential form of the dynamic programming principle. This nonlinear first-order PDE appears to be well-posed in the framework of viscosity solutions introduced by Crandall and Lions , , . These tools also allow to perform the numerical analysis of discretization schemes. The theoretical contributions in this direction did not cease growing, see the books by Barles and Bardi and Capuzzo-Dolcetta .
The so-called direct methods consist in an optimization of the trajectory, after having discretized time, by a nonlinear programming solver that possibly takes into account the dynamic structure. So the two main problems are the choice of the discretization and the nonlinear programming algorithm. A third problem is the possibility of refinement of the discretization once after solving on a coarser grid.
In the full discretization approach, general Runge-Kutta schemes with different values of control for each inner step are used. This allows to obtain and control high orders of precision, see Hager , Bonnans . In an interior-point algorithm context, controls can be eliminated and the resulting system of equation is easily solved due to its band structure. Discretization errors due to constraints are discussed in Dontchev et al. . See also Malanowski et al. .
In the indirect approach, the control is eliminated thanks to Pontryagin's maximum principle. One has then to solve the two-points boundary value problem
(with differential variables state and costate) by a single or multiple shooting method. The questions are here the choice of a discretization scheme for the
integration of the boundary value problem, of a (possibly globalized) Newton type algorithm for solving the resulting finite dimensional problem
in
For state constrained problems or singular arcs, the formulation of the shooting function may be quite elaborate , , . As initiated in , we focus more specifically on the handling of discontinuities, with ongoing work on the geometric integration aspects (Hamiltonian conservation).
This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting when the problem is not convex.
Characterization of the value function >From the dynamic programming principle, we derive a characterization of the value function as being a solution (in viscosity sense) of an Hamilton-Jacobi-Bellman equation, which is a nonlinear PDE of dimension equal to the number n of state variables. Since the pioneer works of Crandall and Lions , , , many theoretical contributions were carried out, allowing an understanding of the properties of the value function as well as of the set of admissible trajectories. However, there remains an important effort to provide for the development of effective and adapted numerical tools, mainly because of numerical complexity (complexity is exponential with respect to n).
Numerical approximation for continuous value function Several numerical schemes have been already studied to treat the case when the solution of the HJB equation (the value function) is continuous. Let us quote for example the Semi-Lagrangian methods , studied by the team of M. Falcone (La Sapienza, Rome), the high order schemes WENO, ENO, Discrete galerkin introduced by S. Osher, C.-W. Shu, E. Harten , , , , and also the schemes on nonregular grids by R. Abgrall , . All these schemes rely on finite differences or/and interpolation techniques which lead to numerical diffusions. Hence, the numerical solution is unsatisfying for long time approximations even in the continuous case.
One of the (nonmonotone) schemes for solving the HJB equation is based on the Ultrabee algorithm proposed, in the case of advection equation with constant velocity, by Roe and recently revisited by Després-Lagoutière , . The numerical results on several academic problems show the relevance of the antidiffusive schemes. However, the theoretical study of the convergence is a difficult question and is only partially done.
Optimal stochastic control problems occur when the dynamical system is uncertain. A decision typically has to be taken at each time, while realizations of future events are unknown (but some information is given on their distribution of probabilities). In particular, problems of economic nature deal with large uncertainties (on prices, production and demand). Specific examples are the portfolio selection problems in a market with risky and non-risky assets, super-replication with uncertain volatility, management of power resources (dams, gas). Air traffic control is another example of such problems.
Nonsmoothness of the value function. Sometimes the value function is smooth (e.g. in the case of Merton's portfolio problem, Oksendal ) and the associated HJB equation can be solved explicitly. Still, the value function is not smooth enough to satisfy the HJB equation in the classical sense. As for the deterministic case, the notion of viscosity solution provides a convenient framework for dealing with the lack of smoothness, see Pham , that happens also to be well adapted to the study of discretization errors for numerical discretization schemes , .
Numerical approximation for optimal stochastic control problems. The numerical discretization of second order HJB equations was the subject of several contributions. The book of Kushner-Dupuis gives a complete synthesis on the Markov chain schemes (i.e Finite Differences, semi-Lagrangian, Finite Elements, ...). Here a main difficulty of these equations comes from the fact that the second order operator (i.e. the diffusion term) is not uniformly elliptic and can be degenerated. Moreover, the diffusion term (covariance matrix) may change direction at any space point and at any time (this matrix is associated the dynamics volatility).
For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold . Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree .
We work with colleagues from U. Chile, in the framework of Inria Chile, on the management of electricity production and storage for a microgid.
We have a collaboration with the startup Safety Line on the optimization of trajectories for civil aircrafts.
We have a collaboration with IFPEN on the energy management for hybrid vehicles.
Web page: http://
The Bocop project aims to develop an open-source toolbox for solving optimal control problems, with collaborations with industrial and academic partners. Optimal control (optimization of dynamical systems governed by differential equations) has numerous applications in transportation, energy, process optimization, and biology. The software reuses some packages from the COIN-OR library, in particular the well-known nonlinear programming solver Ipopt, features a user-friendly interface and can be deployed on Windows / Mac / Linux.
The project is supported by Inria with the recruitment of Vincent Grelard as developer in 2010-2012,
Daphné Giorgi (Oct. 2012-Sept. 2014), and Olivier Tissot since October 2014.
The first prototype was released at the end of 2011, Bocop is currently at version 2.0.1 and has been
downloaded more than 700 times. The software was first succesfully tested on several academic problems,
see available on http://
Bocop auto-assessment according to Inria notice: A3up4, SO3, SM3, EM3up4, SDL4up5
The paper about running strategies proves Keller's conjecture. It was highlighted in SIAM Connect,see
http://
Following the meeting with the startup Safety Line at Imatch "Optimisation and Control" in october 2013, we conducted a first collaboration of six months on optimizing the fuel consumption of civil airliners. This first step successfully established the proof of concept and was validated by actual test flights in June 2014, leading to a shared patent and the development of a specific module of our software 'Bocop', included in the tool 'OptiClimb' developed at Safety Line. Future prospects include improving the numerical robustness of the current tool, as well as expanding the optimization to the cruise flight in addition to the climb phase.
F. Bonnans, with M.S. Aronna (IMPA, Rio de Janeiro) and B.-S. Goh (Curtin U., Miri, Sarawak, Malaysia) obtained in new second order necessary and sufficient optimality conditions for a class of control-affine problems with a scalar control and a scalar state constraint. These optimality conditions extend to the constrained state framework the Goh transform, which is the classical tool for obtaining an extension of the Legendre condition. We propose a shooting algorithm to solve numerically this class of problems and we provide a sufficient condition for its local convergence. We provide examples to illustrate the theory. An article by F. Bonnans, X. Dupuis (Ceremade, U. Dauphine) and L. Pfeiffer (U. Graz) has been published in the SIAM J. Control Optim. on “Second-order necessary conditions in Pontryagin form for optimal control problems” .
With J. Gianatti (U. Rosario) and F. Silva (U. Limoges) we obtained an extension of the Sakawa-Shindo algorithm (for computing a solution of the optimality system of a deterministic optimal control problem) to stochastic control problems. The paper is in progress.
In the framework of the thesis of Nicolas Grébille, we continued our study of decomposition algorithms for a stochastic model of optimal electricity energy production.The energy production is divided in a number of zones. The idea is to constrain the energy flows between these zones, by linear feedback to the demand (wich is a random variable). The coefficients of the feedback are to be optimized. Then the problem is decomposed for each zone (and can then be solved easily by a SDDP type algorithm). We obtained encouraging preliminary numerical results in a three zones problem.
Faisal Wahid developed a mixed integer program model for hydro-power producers participating in the future intra-day French Electricity Balancing Market. He has also formulated the mixed integer stochastic dynamic program model for the more general hydro- bidding under uncertainty. The objective of this model is to produce optimal offer policies in the form of supply curves under a time inhomogeneous Markov process of electricity market clearing prices.
The paper is concerned with stochastic optimal control for a running maximum cost. A direct approach based on dynamic programming techniques is studied leading to the characterization of the value function as the unique viscosity solution of a second order Hamilton- Jacobi-Bellman (HJB) equation with an oblique derivative boundary condition. A general numerical scheme is proposed and a convergence result is provided. Error estimates are obtained for the semi-Lagrangian scheme. These results can apply to the case of lookback options in finance. Moreover, optimal control problems with maximum cost arise in the characterization of the reachable sets for a system of controlled stochastic differential equations. Some numerical simulations on examples of reachable analysis are included to illustrate our approach.
An optimal finite-time horizon feedback control problem for (semi linear) wave equations is studied in . The feedback law can be derived from the dynamic programming principle and requires to solve the evolutionary Hamilton-Jacobi-Bellman (HJB) equation. Classical discretization methods based on finite elements lead to approximated problems governed by ODEs in high dimensional space which makes infeasible the numerical resolution by HJB approach. In the present paper, an approximation based on spectral elements is used to discretize the wave equation. The effect of noise is considered and numerical simulations are presented to show the relevance of the approach
The works , deal
with deterministic control problems where the dynamic and the
running cost can be completely different in two (or more)
complementary domains of the space
This work deals with a state-constrained control problem. It is well known that, unless some compatibility condition between constraints and dynamics holds, the value function has not enough regularity, or can fail to be the unique constrained viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. Here, we consider the case of a set of constraints having a strati
ed structure. Under this circumstance, the interior of this set may be empty or disconnected, and the admissible trajectories may have the only option to stay on the boundary without possible approximation in the interior of the constraints. In such situations, the classical pointing quali
cation hypothesis are not relevant. The discontinuous Value Function is then characterized by means of a system of HJB equations on each stratum that composes the state-constraints. This result is obtained under a local controllability assumption which is required only on the strata where some chattering phenomena could occur.
We investigate in convex constrained nonlinear optimization problems and optimal control with convex state constraints. For this purpose we endow the interior of constraints set with the structure of Riemannian manifold. In particular, we consider a class of Riemannian metric induced by the squared Hessian of a Legendre functions. We describe in details the geodesic curves on this manifolds and we propose a gradient-like algorithm for constrained optimization based on linear search along geodesics. We also use the Legendre change of coordinates to study the Value Function of a Mayer problem with state constraints. We provide a characterization of the Value Function for this problem as the unique viscosity solution of the Hamilton-Jacobi-Bellman equation.
In the paper we study state-constrained discontinuous ordinary differential equations for which the corresponding vector field has a set of singularities that forms a stratification of the state domain. Existence of solutions and robustness with respect to external perturbations of the righthand term are investigated. Moreover, notions of regularity for stratifications are discussed.
Stabilization of Burgers equation to nonstationary trajectories A. Kröner and Sérgio S. Rodrigues (RICAM, Linz, Austria) considered in using infinite-dimensional internal controls. Estimates for the dimension of the controller are derived; in the particular case of no constraint in the support of the control a better estimate is derived and the possibility of getting an analogous estimate for the general case is discussed. Numerical examples are presented illustrating the stabilizing effect of the feedback control, and suggesting that the existence of an estimate in the general case analogous to that in the particular one is plausible. In the problem was consided for a finite number of internal piecewise constant controls.
Reduced-order minimum time control of advection-reaction -diffusion systems via dynamic programming Dante Kalise (RICAM, Linz, Austria) and A. Kröner considered in . The authors use balanced truncation for the model reduction part and include a Luenberger observer.
A semi-Lagrangian scheme for
The minimum effort problem for the wave equation K. Kunisch (University of Graz,
Austria) and A. Kröner considered in .
The problem involves
In collaboration with team McTAO (Sophia), we started in 2013 to study the contrast imaging problem in nuclear magnetic resonance, modeled as Mayer problem in optimal control ( ). Using tools from the Maximum Principle and geometric control, we obtained a first synthesis of locally optimal solutions is given in the single-input case, as well as preliminary results in the bi-input case. This analysis was supported by comprehensive numerical investigations using a combination of indirect shooting (HamPath software) and direct method (Bocop), with a moment-based (LMI) technique to estimate the global optimum.
These results have been extended in 2014, on the theoretical side with the classification of singular extremals (), and on the numerical side with the study of a large number of spins particles subject to spatial inhomogeneities in the magnetic field.
Following , we pursued the study of the N-link swimmer, a generalization of the classical Purcell swimmer. We use the model of the Resistive Force Theory to derive the motion equation for the swimmer in a fluid with a low Reynolds number. This allows use to study and solve the optimal swimming problem in the framework of optimal control. We extend our previous study of the optimal strokes by moving to the optimal design of the swimmer. In we provide an estimate of the optimal link ratio for maximal displacement, based on an expansion for small amplitudes. This theoretical result is supported by numerical simulations, that also give some insight on the type of optimal strokes depending on the constraints on the amplitude and deformation speed.
We study the energy management problem for a microgrid including a diesel generator and a photovoltaic plant with a battery storage system. The objective is to minimize the total operational cost over a certain timeframe, primarily the diesel consumption, while satisfying a prescribed power load. After reformulation, the decision variables can be reduced to the charging/discharging power for the battery system. We take into account the switching cost for the diesel generator, the non-convex objective, and the long-term aging of the batteries. We solve this problem using a continuous optimal control framework, with both a direct transcription method (time discretization) and a Dynamic Programming method (Hamilton Jacobi Bellman). This project is a collaboration between team COMMANDS (Inria Saclay, France) and Centro de Energia (Universidad de Chile, Chile). A first paper is currently in preparation, while ongoing studies include comparison with the existing MILP approach, more refined battery aging models, and modeling the stochastic nature of the photovoltaic power and power load.
EDF, PhD thesis of N. Grébille, 'Numerical methods for solving stochastic equilibrium problems'.
IFPEN, PhD thesis of F.Bleuse, 'Optimal control and robustness for rechargeable hybrid vehicles'. The study is focused on the so-called parallel architecture, with both the thermal and electric engines able to move the vehicle. The main axis is to optimize the use of the thermal engine.
Safety Line (startup in aeronautics), research and transfer contract, optimization of fuel consumption for civil planes. A first part is devoted to the identification of the aerodynamic and thrust characteristics of the plane, using recorded flight data. A second part is optimizing the fuel consumption during the climb phase.
F. Bonnans is coordinator of the ICODE project “Strategic crowds: analysis and simulation”, with U. Limoges, Paris-Sud.
F. Bonnans participates to two PGMO project: “Hydro-electric scheduling under uncertainty”, with U. Auckland, “Perturbation analysis for deterministic and stochastic optimal control problems”, with U. Limoges and TSE (Toulouse),
P. Martinon participates to the OPTIBIO project, supported by FMJH-PGMO, devoted to “New challenges in the optimal control of bioprocesses”, with U. Angers, Lille 1 / Limoges
This project is a collaboration in the framework of a 3-year (2012-2015) research program funded by DGA. The title of the project is “Problèmes de commande optimale pour des systèmes non-linéaires en présence d’incertitudes et sous contraintes de probabilité de succès”.
The team is part of the collaborative project HJNet funded by the French National Research Agency
(ANR-12-BS01-0008-01). It started in January 2013 and will end in December 2013.
Website: http://
Type: FP7
Defi: NC
Instrument: Initial Training Network
Objectif: NC
Duration: January 2011 - December 2014
Coordinator: Inria
Partner: Univ. of Louvain, Univ. Bayreuth, Univ. Porto, Univ. Rome - La Sapienza, ICL, Astrium-Eads, Astos solutions, Volkswagen, Univ. Padova, Univ. Pierre et Marie Curie.
Inria contact: Hasnaa Zidani
Abstract: Optimisation-based control systems concern the determination of control strategies for complex, dynamic systems, to optimise some measures of best performance. It has the potential for application to a wide range of fields, including aerospace, chemical processing, power systems control, transportation systems and resource economics. It is of special relevance today, because optimization provides a natural framework for determining control strategies, which are energy efficient and respect environmental constraints. The multi-partner initial training network SADCO aims at: Training young researchers and future scientific leaders in the field of control theory with emphasis on two major themes sensitivity of optimal strategies to changes in the optimal control problem specification, and deterministic controller design; Advancing the theory and developing new numerical methods; Conveying fundamental scientific contributions within European industrial sectors.
We are involved in the CIRIC team “Optimization and control of energy”, jointly with U. de Chile at Santiago. This collaboration involved several visits of the team in Santiago: F. Bonnans (1 week), B. Heymann (2 months) and P. Martinon (2 weeks).
Title: Optimization and control in network economics
International Partner:
Universidad de Chile (CHILI)
Duration: 2012 - 2014.
See also: http://
Limited resources in telecommunication, energy, gas and water supply networks, lead to multi-agent interactions that can be seen as games or economic equilibrium involving stochastic optimization and optimal control problems. Interaction occurs within a network, where decisions on what to produce, consume, trade or plan, are subject to constraints imposed by node and link capacities, risk, and uncertainty, e.g. the capacity of generators and transmission lines; capacity of pipeline in gas supply; switches and antennas in telecommunication. At the same time, nonlinear phenomena arise from price formation as a consequence of demand-supply equilibria or multi-unit auction processes in the case of energy and telecommunication. We will focus first in this project in electricity markets in which there are producers/consumers PCs, and an agent called ISO (Independent system operator) in charge of the management of the network. One major application we have in mind is the one of smart (electrical) grids, in view of the increased use of renewable energies, that is, a massive entry of wind, geothermal, solar in particular.
Collaboration with the CIFASIS lab of U. Rosario, Argentina (3 months visit of J. Gianatti).
Andew Philpott, U. Auckland (NZ), 6 weeks. Research on stochastic optimization with F. Bonnans and F. Wahid.
Justina Gianatti, Cifasis, U. Rosario (Argentina), 3 months. Research on stochastic control with F. Bonnans.
A. Kröner: Local organizer of the Workshop “Geometric control and related fields” within the Special Semester on New Trends in Calculus of Variations, Linz, Austria, 17.11.-21.11.2014.
H. Zidani: NETCO'2014, June 23-27, 2014, Tours.
F. Bonnans: PGMO-COPI'14 conf., October 28-31, 2014, Ecole Polytechnique, Palaiseau.
F. Bonnans: EUROPT Workshop on Advances in Continuous Optimization, 10-12 July, 2014, Perpignan.
F. Bonnans is Corresponding Editor of “ESAIM:COCV” (Control, Optimization and Calculus of Variations), and Associate Editor of “Applied Mathematics and Optimization”, “Optimization, Methods and Software”, and “Series on Mathematics and its Applications, Annals of The Academy of Romanian Scientists”.
H. Zidani is Associate Editor of “Set-Valued and Variational Analysis”, and "Communication of Pure and Applied Analysis".
Master
F. Bonnans: Numerical analysis of partial differential equations arising in finance and stochastic control, 24h, M2, Ecole Polytechnique and U. Paris 6, France.
F. Bonnans: Optimal control, 10h, M2, Ensta, France.
H. Zidani: Optimal control, 10h, M2, Ensta, France.
E-learning
F. Bonnans, several lecture notes on the page
PhD in progress : Florine Bleuse, Optimal control and robustness for rechargeable hybrid vehicles. Started October 2013, F. Bonnans and P. Martinon, IFPEN fellowship.
PhD in progress : Benjamin Heymann, Dynamic optimization with uncertainty; application to energy production. Started October 2013, F. Bonnans, Polytechnique fellowship.
PhD in progress : Nicolas Grébille, Numerical methods for solving stochastic equilibrium problems; application to energy markets. Started January 2013, F. Bonnans, CIFRE fellowship (EDF R & D).
PhD in progress: Athena Picarelli, First and Second Order Hamilton-Jacobi equations for State-Constrained Control Problems. Started November 2011, O. Bokanowski and H. Zidani, SADCO fellowship.
PhD in progress: Cristopher Hermosilla, Feedback controls and optimal trajectories. Started November 2011, H. Zidani, SADCO fellowship.
PhD in progress: Mohamed Assellaou, Reachability analysis for stochastic controlled systems. Started October 2011, O. Bokanowski and H. Zidani.
The paper about running strategies was reported in several media in 2014, especially by Figaro (May 29), Usine Nouvelle (May 18), Science Daily (May 15), MedicalXPress (May 15), Ipodiatry (May 21) and French Huffington Post in 2013.