Commands is shared on both the CMAP (Ecole Polytechnique) and the UMA (ENSTA).

Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.

Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.

For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses or of posdocts. The Commands team itself has dealt since its foundation in 2007 with several types of applications:

Space vehicle trajectories, in collaboration with CNES, the French space agency,

Production, management, storage and trading of energy resources (in collaboration with EDF, GDF and TOTAL).

Energy management for hybrid vehicles (in collaboration with Renault).

We give more details in the Application domain section.

The article “Deterministic state constrained optimal control problems without controllability assumptions”, by O. Bokanowski, N. Forcadel and H. Zidani, was “Highlight Paper” of the issue 17-04 (October 2011) of the ESAIM-COCV journal.

The BOCOP toolbox for optimal control released its first version: see
http://

Commands team has been successful in the Marie Curie (EU) award on the project "SADCO-Sensitivity Analysis for Deterministic Controller Design", under the 7th Framework Programme “FP7-PEOPLE-2010-ITN” Grant agreement number 264735-SADCO. The project is funded 5.6MEuros over 4 years (January 2011-December 2014).

The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in ), with improvements due to the “Chicago school”, Bliss during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young ).

*Trajectory optimization*really started with the spectacular achievement done by Pontryagin's group
during the fifties, by stating, for general optimal control
problems, nonlocal optimality conditions generalizing those of Weierstrass. This motivated the application to many industrial problems (see the classical books by Bryson and Ho
, Leitmann
, Lee and Markus
, Ioffe and Tihomirov
). Since then, various theoretical achievements have been obtained
by extending the results to nonsmooth problems, see Aubin
, Clarke
, Ekeland
.

*Dynamic programming*was introduced and systematically studied by R. Bellman during the fifties. The HJB equation, whose solution is the value function of the (parameterized) optimal
control problem, is a variant of the classical Hamilton-Jacobi equation of mechanics for the case of dynamics parameterized by a control variable. It may be viewed as a differential form of the
dynamic programming principle. This nonlinear first-order PDE appears to be well-posed in the framework of
*viscosity solutions*introduced by Crandall and Lions
,
,
. These tools also allow to perform the numerical analysis of
discretization schemes. The theoretical contributions in this direction did not cease growing, see the books by Barles
and Bardi and Capuzzo-Dolcetta
.

The so-called
*direct methods*consist in an optimization of the trajectory, after having discretized time, by a nonlinear programming solver that possibly takes into account the dynamic structure. So
the two main problems are the choice of the discretization and the nonlinear programming algorithm. A third problem is the possibility of refinement of the discretization once after solving on
a coarser grid.

In the
*full discretization approach*, general Runge-Kutta schemes with different values of control for each inner step are used. This allows to obtain and control high orders of precision, see
Hager
, Bonnans
. In an interior-point algorithm context, controls can be
eliminated and the resulting system of equation is easily solved due to its band structure. Discretization errors due to constraints are discussed in Dontchev et al.
. See also Malanowski et al.
.

In the
*indirect*approach, the control is eliminated thanks to Pontryagin's maximum principle. One has then to solve the two-points boundary value problem (with differential variables state and
costate) by a single or multiple shooting method. The questions are here the choice of a discretization scheme for the integration of the boundary value problem, of a (possibly globalized)
Newton type algorithm for solving the resulting finite dimensional problem in

For state constrained problems the formulation of the shooting function may be quite elaborated , . As initiated in , we focus more specifically on the handling of discontinuities, with ongoing work on the geometric integration aspects (Hamiltonian conservation).

This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting, when the problem is not convex.

*Characterization of the value function*From the dynamic programming principle, we derive a characterization of the value function as being a solution (in viscosity sense) of an
Hamilton-Jacobi-Bellman equation, wich is a nonlinear PDE of dimension equal to the number n of state variables. Since the pioneer works of Crandall and Lions
,
,
, many theoretical contributions were carried out, allowing an
understanding of the properties of the value function as well as of the set of admissible trajectories. However, there remains an important effort to provide for the development of effective
and adapted numerical tools, mainly because of numerical complexity (complexity is exponential with respect to n).

*Numerical approximation for continuous value function*Several numerical schemes have been already studied to treat the case when the solution of the HJB equation (the value function) is
continuous. Let us quote for example the Semi-Lagrangian methods
,
studied by the team of M. Falcone (La Sapienza, Rome), the high
order schemes WENO, ENO, Discrete galerkin introduced by S. Osher, C.-W. Shu, E. Harten
,
,
,
, and also the schemes on nonregular grids by R. Abgrall
,
. All these schemes rely on finite differences or/and interpolation
techniques which lead to numerical diffusions. Hence, the numerical solution is unsatisfying for long time approximations even in the continuous case.

One of the (nonmonotone) schemes for solving the HJB equation is based on the Ultrabee algorithm proposed, in the case of advection equation with constant velocity, by Roe and recently revisited by Després-Lagoutière , . The numerical results on several academic problems show the relevance of the antidiffusive schemes. However, the theoretical study of the convergence is a difficult question and is only partially done.

*Optimal stochastic control problems*occur when the dynamical system is uncertain. A decision typically has to be taken at each time, while realizations of future events are unknown (but
some information is given on their distribution of probabilities). In particular, problems of economic nature deal with large uncertainties (on prices, production and demand). Specific examples
are the portfolio selection problems in a market with risky and non-risky assets, super-replication with uncertain volatility, management of power resources (dams, gas). Air traffic control is
another example of such problems.

*Nonsmoothness of the value function*. Sometimes the value function is smooth (e.g. in the case of Merton's portfolio problem, Oksendal
) and the associated HJB equation can be solved explicitly. Still,
the value function is not smooth enough to satisfy the HJB equation in the classical sense. As for the deterministic case, the notion of viscosity solution provides a convenient framework for
dealing with the lack of smoothness, see Pham
, that happens also to be well adapted to the study of
discretization errors for numerical discretization schemes
,
.

*Numerical approximation for optimal stochastic control problems*. The numerical discretization of second order HJB equations was the subject of several contributions. The book of
Kushner-Dupuis
gives a complete synthesis on the chain Markov schemes (i.e Finite
Differences, semi-Lagrangian, Finite Elements, ...). Here a main difficulty of these equations comes from the fact that the second order operator (i.e. the diffusion term) is not uniformly
elliptic and can be degenerated. Moreover, the diffusion term (covariance matrix) may change direction at any space point and at any time (this matrix is associated the dynamics
volatility).

For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold . Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree .

Commands is a team with a strong commitment in tackling real-life applications in addition to theoretical challenges. This shows in our long history of contracts with industrial partners. In the recent years, we have mainly contributed to the following fields of application.

In the framework of a long-term partnership with the Cnes, and more recently Astrium, we have studied trajectory optimization for space launcher problems. This kind of problems typically involves hard constraints (thermal flux, mechanical efforts) and inexact models (atmosphere, aerodynamic forces). The two main achievments were to study when singular arcs may occur, and to show the effectiveness of a HJB approach on a reduced model. Singular arcs are flight phases with a non-maximal thrust, induced by a tradeoff between speed and atmospheric drag; they cause difficulties of both theoretical and practical nature. The latter point is the first step in the process of applying global methods to this class of difficult problems.

In a partnership with Total, we have studied problems dealing with the trading of Liquefied Natural Gas. We have computed maximizing revenue policies, by combining the Stochastic Dual Dynamic Programmin approach (SDDP) with a quantization method for the noise that enters in prices. We have also given partial results for the case of integer decision.

With Renault, we have studied problems of energy management for hybrid vehicles. Hybrid vehicles include an auxiliary thermal (gas) engine that is used as a range extender for the main electric propulsion. We are interested in determining the optimal policies for energy management, taking into account some stochatic incertainties, as well as execution delay and decision lags.

The Bocop project aims to develop an open-source toolbox for solving optimal control problems, with collaborations with industrial and academic partners. Optimal control (optimization of dynamical systems governed by differential equations) has numerous applications in transportation, energy, process optimization, and biology. This project is supported by INRIA in the framework of an ADT, Action de Développement Technologique, 2010-2012.

The software uses some packages from the COIN-OR library, in particular the well-known interior-point nonlinear programming solver Ipopt. It also features a user-friendly interface in Scilab.

See the web page
http://

This project aims at developping sequential and parallel MPI/openMP C++ solvers for the approximation of Hamilton-Jacobi-Bellman (HJB) equations in a d-dimensional space. The main goal is to provide an HJB solvers that can work in dimension d (limited by the machine's capacity). The solver outputs can be visualized with Matlab or Paraview (via VTK files).

The HJB Solver has been actively developed under a partnership between COMMANDS and the SME HPC-project in the period between December 2009 to November 2011. See also
http://

We release two versions:

HJB-SEQUENTIAL-REF: sequential version that can run on any machine

HJB-PARALLEL-REF: parallel version that can run only on multi-core architectures.

Shoot was designed for the resolution of optimal control problems via indirect methods (necessary conditions, Pontriagyn's Maximum Principle). Such methods transform the original problem into finding a zero of a certain shooting function. The package offers several choices of integrators and solvers, and can handle control discontinuities. Features also include the use of variational equations to compute the Jacobian of the shooting function, as well as homotopy and grid shooting techniques for easier initialization.

See also the web page
http://

These studies enter in the framework of the PhD thesis of S. Aronna, supervised by J.F. Bonnans and P. Lotito, that ended in December 2011.

In , we study an optimal control problem governed by measure driven differential systems and in presence of state constraints. First, under some weak invariance assumptions, we study in the properties of the value function and obtain its characterization by means of an auxiliary control problem of absolutely continuous trajectories. For this, we use some known techniques of reparametrization and graph completion. Then we give a characterization of the value function as the unique constrained viscosity solution of a Hamilton-Jacobi equation with measurable time dependant Hamiltonians.

The general case without assuming any controllability assumption is considered in
. We prove that the optimal solutions can still be obtained by
solving a reparametrized control problem of absolutely continuous trajectories but with
**time-dependent state-constraints**.

The paper deals with minimal time problems governed by nonlinear systems under general time dependent state constraints and in the two-player games setting. In general, it is known that the characterization of the minimal time function, as well as the study of its regularity properties, is a difficult task in particular when no controllability assumption is made. In addition to these difficulties, we are interested here to the case when the target, the state constraints and the dynamics are allowed to be time-dependent.s We introduce a particular reachability control problem, which has a supremum cost function but is free of state constraints. This auxiliary control problem allows to characterize easily the backward reachable sets, and then, the minimal time function, without assuming any controllability assumption. These techniques are linked to the well known level-set approaches. Our results can be used to deal with motion planning problems with obstacle avoidance, see .

Several works have been also carried out in the domain of numerical methods of HJB equations. The paper aims at studing a discontinuous Galerkin scheme for front propagation with obstacles. We extend a first work published in , to propose a simple and direct discontinuous Galerkin (DG) method adapted to such front propagation problems. We follow the formulation of , leading to a level set formulation driven by a Hamilton-Jacobi variational inequality. The DG scheme is motivated by the variational formulation when the equation corresponds to linear convection problems in presence of obstacles. The scheme is then generalized to nonlinear equations, written in an explicit form. Stability analysis are performed for the linear case with Euler forward, a Heun scheme and a Runge-Kutta third order time discretization. Several numerical examples are provided to demonstrate the robustness of the method. Finally, a narrow band approach is considered in order to reduce the computational cost.

INRIA - RENAULT,
*Energy management for hybrid vehicles*, PhD. fellowship of G. Granato, Dec 2009 - Nov 2012. Involved researchers: F. Bonnans,
**H. Zidani**.

INRIA - TOTAL,
*Trading of Liquefied Natural Gas*, PhD fellowship (CIFRE) of Y. Cen, Dec 2008 - Dec 2011. Involved researchers:
**F. Bonnans**.

INRIA - HPC PROJECT,
*Bibliothèque numérique de calcul parallèle: Equations HJB*, Dec 2009 - Dec 2011. Involved researchers: O. Bokanowski, F. Bonnans, N. Forcadel,
**H. Zidani**.

We participate to the DIM-Digiteo Alma project. This research project deals with Acute Myelogenous Leukaemia (AML), its mechanisms, controlled by molecular events at the DNA level, and its treatments. See the DISCO team activity report for more details. We are preparing optimal control tools for analyzing the models.

Our research activities in Hamilton Jacobi approach for state-constrained control problems is supported by a DGA grant (DGA-ENSTA No 0660037). We are applying our results on reachability analysis and motion planning for collision avoidance for UAVs.

Title: Sensitivity Analysis for Deterministic Controller Design

Type: PEOPLE F7

Instrument: Initial Training Network (ITN)

Duration: January 2011 - December 2014

Coordinator: INRIA (France)

See also:
http://

Abstract: Optimisation-based control systems concern the determination of control strategies for complex, dynamic systems, to optimise some measures of best performance. It has the potential for application to a wide range of fields, including aerospace, chemical processing, power systems control, transportation systems and resource economics. It is of special relevance today, because optimization provides a natural framework for determining control strategies, which are energy efficient and respect environmental constraints.

The multi-partner initial training network SADCO aims at: Training young researchers and future scientific leaders in the field of control theory with emphasis on two major themes sensitivity of optimal strategies to changes in the optimal control problem specification, and deterministic controller design; Advancing the theory and developing new numerical methods; Conveying fundamental scientific contributions within European industrial sectors.

In order to reach these objectives, SADCO establishes a collaborative research and training network of 11 full partners from both the academic and industrial sectors, and gathers participants with expertises in complementary disciplines in mathematics and engineering. The network also offers a complete range of theoretical, practical and complementary training as well as scientific workshops. SADCO will work together with the young researchers to develop and implement effective training plans tailored to each individual requirements. Multi-disciplinary training based on the integrated scientific programme, secondments, regular meetings, active networking, will ensure the success of this projects.

The development of new 'clean' technologies in power, transportation and other domains is a major opportunity for EU industries. The research programme will help place EU universities in the forefront of Optimal Control, a field of mathematics that supports these technologies.

The training programme, based on institutions covering the principal areas of the field, will provide a new generation of young mathematicians with broad skills in Optimal Control, which are not readily acquired at one institution alone. They will be equipped to take forward research in Optimal Control at universities, or to work in related, emerging technological areas, of vital importance to society.

Peter Wolenski, Luisiane State University (4 months, August to November 2011)

Roberto Ferretti, University of Rome 3 (1 week, August 2011)

Alejandro Jofre, University of Chile (2 weeks, July 2011)

Elina Mancinelli, University of Rosario (3 weeks, April 2011)

Antonio Siconolfi, University of Rome 1 - La Sapienza (2 weeks, May 2011)

Mohamed Mnif, ENIT Tunis (2 weeks, May 2011)

Pablo Lotito, University of Rosario (1 week, December 2011)

Soledad Aronna (from Feb 2011 until Dec 2011)

Subject: Optimal control of systems with singular arcs

Institution: CONICET (Argentina)

Imène Ben Latifa (from Feb 2011 until Apr 2011)

Subject: Optimal multiple stopping and valuation of swing options with jumps

Institution: Ecole Nationale d'Ingénieurs de Tunis (Tunisia)

Eduardo Philip (from Apr 2011 until Jul 2011)

Subject: Optimal control problems of BV trajectories and with state constraints

Institution: Universidad Nacional de Rosario (Argentina)

We are setting up a project with Alejandro Jofre (U. Chile, Santiago) in the framework of the CIRIC initiative, on the subject of smart grid optimization.

F. Bonnans is Corresponding Editor of “ESAIM:COCV” (Control, Optimization and Calculus of Variations), and Associate Editor of “Applied Mathematics and Optimization”, and “Optimization, Methods and Software”.

F. Bonnans is chairman of the SMAI-MODE group (the optimization group of the French Applied Mathematics Society).

Teaching

Frédéric Bonnans: Cochair of the third year program in applied mathematics, Ecole Polytechnique, France.

Master Continuous Optimization, 18h, M2, Ecole Polytechnique and U. Paris 6, France.

Master Numerical analysis of partial differential equations arising in finance and stochastic control, 24h, M2, Ecole Polytechnique and U. Paris 6, France.

Hasnaa Zidani: in charge of the third year module "B7: Commande des Systèmes" (84h) at Ensta ParisTech, and of the following courses:

Master Numerical approximation for front propagation, 21h, M2, Ensta ParisTech, France (also at ECP, France, in collaboration with Ch. Chalons).

Master Optimal control of nonlinear systems, 21h, M2, Ensta ParisTech

License Quadratic Optimization, L1, 21h, Ensta ParisTech, France.

Pierre Martinon: Teaching Assistant

License Quadratic Optimization, L1, 12h, Ensta ParisTech, France.

License Matlab Introduction, L1, 16h, Ensta ParisTech, France.

License Numerical Analysis, L1, 16h, Ensae, France.

Zhiping Rao: teaching assistant

Licence Quadratic Optimisation, L1, 16h, ENSTA ParisTech, France.

Master Markov Chain process, M1, 16h, ENSTA ParisTech, France.

PhD & HdR theses

HdR : Nicolas FORCADEL, Contribution à l'analyse d'équations aux dérivées partielles avec applications à la dynamique des dislocations et au contrôle optimal. Université Paris-Dauphine, 05-12-2011.

PhD : Maria Soledad ARONNA, Second order analysis of optimal control problems with singular arcs. Optimality conditions and shooting algorithm. Ecole Polytechnique, 15-12-2011.

PhD : Zhihao CEN, LNG portfolio optimization; approach by stochastic programming techniques. Ecole Polytechnique, 22-11-2011.

PhD in progress : Imène Ben-Latifa, Optimal multiple stopping and valuation of swing options in jump models. Oct. 2010, F. Bonnans and M. Mnif (ENIT, Tunis).

PhD in progress : Xavier Dupuis, Optimal control of populations; medical applications. Sept. 2010, F. Bonnans.

PhD in progress : Giovanni Granato, Energy management for an electric vehicle with range extender. January 2010, F. Bonans and H. Zidani.

PhD in progress : Laurent Pfeiffer, Optimal control of large electrical networks. Sept. 2010, F. Bonnans.

PhD in progress: Zhiping Rao, Hamilton-Jacobi equations with discontinuous coefficients. Sept. 2010, H. Zidani and N. Forcadel.

PhD in progress: Athena Picarelli, First and Second Order Hamilton-Jacobi equations for State-Constrained Control Problems. Nov. 2011, O. Bokanowski and H. Zidani

PhD in progress: Cristopher Hermosilla, Feedback controls and optimal trajectories. Nov. 2011, H. Zidani.

PhD in progress: Mohamed Assellaou, Reachability analysis for stochastic controlled systems. Oct. 2011, O. Bokanowski and H.Zidani.