Commands is a team devoted to dynamic optimization, both for deterministic and stochastic systems. This includes the following approaches: trajectory optimization, deterministic and stochastic optimal control, stochastic programming, dynamic programming and Hamilton-Jacobi-Bellman equation.

Our aim is to derive new and powerful algorithms for solving numerically these problems, with applications in several industrial fields. While the numerical aspects are the core of our approach it happens that the study of convergence of these algorithms and the verification of their well-posedness and accuracy raises interesting and difficult theoretical questions, such as, for trajectory optimization: qualification conditions and second-order optimality condition, well-posedness of the shooting algorithm, estimates for discretization errors; for the Hamilton-Jacobi-Bellman approach: accuracy estimates, strong uniqueness principles when state constraints are present, for stochastic programming problems: sensitivity analysis.

For many years the team members have been deeply involved in various industrial applications, often in the framework of PhD theses or of postdocs. The Commands team itself has dealt since its foundation in 2007 with several types of applications:

Space vehicle trajectories, in collaboration with CNES, the French space agency.

Aeronautics, in collaboration with the startup Safety Line.

Production, management, storage and trading of energy resources (in collaboration with EDF, GDF and TOTAL).

Energy management for hybrid vehicles (in collaboration with Renault).

We give more details in the Bilateral contracts section.

In collaboration with L. Giraldi and M. Zopello, we started in 2013 to study the optimal swimming strategies for micro-swimmers. Our approach allows us to solve the optimal control problem without making restrictive assumptions on the shape of the swimming movements. The first numerical results on the 3-link swimmer indicate the existence of a periodic stroke with a better displacement speed than the canonical stroke presented by Purcell in 1977. Further directions include optimal design of micro-swimmers and comparing our simulations to the movement of live micro-organisms.

In collaboration with CNES, a trajectory optimization problem for Ariane 5 was studied and analyzed by HJB approach. In this study, the flight model is considered in dimension 6 without simplification. The problem consists in maximizing the payload to steer the launcher from the launch base (Kourou) to the GEO orbit. The mission includes ballistic phases and the optimization also encompasses the intermediate GTO orbit parameters. The optimization criterion is the mass of the payload to be injected on the GEO.

Finally, the team completed 3 PhD and 4 patents in 2013.

The roots of deterministic optimal control are the “classical” theory of the calculus of variations, illustrated by the work of Newton, Bernoulli, Euler, and Lagrange (whose famous multipliers were introduced in ), with improvements due to the “Chicago school”, Bliss during the first part of the 20th century, and by the notion of relaxed problem and generalized solution (Young ).

*Trajectory optimization* really started with the
spectacular achievement done by Pontryagin's group
during the fifties, by stating, for general optimal control
problems, nonlocal optimality conditions generalizing those of Weierstrass.
This motivated the application to many industrial problems
(see the classical books by Bryson and Ho ,
Leitmann , Lee and Markus ,
Ioffe and Tihomirov ).
Since then, various theoretical achievements have been obtained
by extending the results to nonsmooth problems, see Aubin ,
Clarke , Ekeland .

*Dynamic programming* was introduced and systematically studied by
R. Bellman during the fifties. The HJB equation, whose solution is the
value function of the (parameterized) optimal control problem,
is a variant of the classical Hamilton-Jacobi equation of mechanics
for the case of dynamics parameterized by a control variable.
It may be viewed as a differential form of the dynamic programming principle.
This nonlinear first-order PDE appears to be well-posed in the framework of
*viscosity solutions* introduced by Crandall and Lions
, , . These tools also allow to perform the
numerical analysis of discretization schemes.
The theoretical contributions in this direction
did not cease growing, see the books by Barles and
Bardi and Capuzzo-Dolcetta .

The so-called *direct methods* consist in an optimization of the trajectory,
after having discretized time, by a nonlinear programming solver that possibly
takes into account the dynamic structure.
So the two main problems are the choice of the discretization and the
nonlinear programming algorithm.
A third problem is the possibility of refinement of the discretization
once after solving on a coarser grid.

In the *full discretization approach*, general
Runge-Kutta schemes with different values of control for each inner
step are used. This allows to obtain and control high
orders of precision, see Hager , Bonnans .
In an interior-point algorithm context, controls can be eliminated and the
resulting system of equation is easily solved due to its band structure.
Discretization errors due to constraints are discussed in
Dontchev et al. .
See also Malanowski et al. .

In the *indirect* approach, the control is eliminated
thanks to Pontryagin's
maximum principle.
One has then to solve the two-points boundary value problem
(with differential variables state and costate) by a single or multiple shooting method.
The questions are here the choice of a discretization scheme for the integration of the boundary value problem, of a
(possibly globalized) Newton
type algorithm for solving the resulting finite dimensional problem
in

For state constrained problems or singular arcs, the formulation of the shooting function may be quite elaborate , , . As initiated in , we focus more specifically on the handling of discontinuities, with ongoing work on the geometric integration aspects (Hamiltonian conservation).

This approach consists in calculating the value function associated with the optimal control problem, and then synthesizing the feedback control and the optimal trajectory using Pontryagin's principle. The method has the great particular advantage of reaching directly the global optimum, which can be very interesting, when the problem is not convex.

*Characterization of the value function*
From the dynamic programming principle, we derive a characterization of the value function
as being a solution (in viscosity sense) of an Hamilton-Jacobi-Bellman equation, which is a nonlinear PDE of dimension equal to the number n of state variables.
Since the pioneer works of Crandall and Lions , , , many theoretical contributions were carried out, allowing an understanding of the properties
of the value function as well as of the set of admissible trajectories.
However, there remains an important effort to provide for the
development of effective and adapted numerical tools, mainly because of numerical complexity (complexity is exponential with respect to n).

*Numerical approximation for continuous value function*
Several numerical schemes have been already studied to treat the case when the solution of the HJB equation (the value function) is continuous. Let us quote for example
the Semi-Lagrangian methods , studied by the team of M. Falcone (La Sapienza, Rome), the high order schemes WENO,
ENO, Discrete galerkin introduced by S. Osher, C.-W. Shu, E. Harten
, , , , and also the schemes on nonregular grids by
R. Abgrall , .
All these schemes rely on finite differences or/and interpolation techniques
which lead to numerical diffusions. Hence, the
numerical solution is unsatisfying for long time approximations even
in the continuous case.

One of the (nonmonotone) schemes for solving the HJB equation is based on the Ultrabee algorithm proposed, in the case of advection equation with constant velocity, by Roe and recently revisited by Després-Lagoutière , . The numerical results on several academic problems show the relevance of the antidiffusive schemes. However, the theoretical study of the convergence is a difficult question and is only partially done.

*Optimal stochastic control problems* occur when the dynamical system is
uncertain. A decision typically has to be taken at each time, while
realizations of future events are unknown (but some information is given
on their distribution of probabilities).
In particular, problems of economic nature deal with large uncertainties
(on prices, production and demand).
Specific examples are the portfolio selection problems in a market with risky
and non-risky assets, super-replication with uncertain volatility,
management of power resources (dams, gas).
Air traffic control is another example of such problems.

*Nonsmoothness of the value function*.
Sometimes the value function is smooth (e.g. in the case
of Merton's portfolio problem, Oksendal )
and the associated HJB equation can be solved explicitly.
Still, the value function is not smooth enough to satisfy the
HJB equation in the classical sense. As for the deterministic
case, the notion of viscosity solution provides a convenient framework for
dealing with the lack of smoothness, see Pham ,
that happens also to be
well adapted to the study of discretization errors for numerical
discretization schemes , .

*Numerical approximation for optimal stochastic control problems*.
The numerical discretization of second order HJB equations was the subject of several contributions.
The book of Kushner-Dupuis gives a complete synthesis on the Markov chain schemes
(i.e Finite Differences, semi-Lagrangian, Finite Elements, ...).
Here a main difficulty of these equations comes from the fact that the second order operator
(i.e. the diffusion term) is not uniformly elliptic and can be degenerated.
Moreover, the diffusion term (covariance matrix) may change direction at any space point and at any time (this matrix is associated the dynamics volatility).

For solving stochastic control problems, we studied the so-called Generalized Finite Differences (GFD), that allow to choose at any node, the stencil approximating the diffusion matrix up to a certain threshold . Determining the stencil and the associated coefficients boils down to a quadratic program to be solved at each point of the grid, and for each control. This is definitely expensive, with the exception of special structures where the coefficients can be computed at low cost. For two dimensional systems, we designed a (very) fast algorithm for computing the coefficients of the GFD scheme, based on the Stern-Brocot tree .

Web page: http://

The Bocop project aims to develop an open-source toolbox for solving optimal control problems, with collaborations with industrial and academic partners. Optimal control (optimization of dynamical systems governed by differential equations) has numerous applications in transportation, energy, process optimization, and biology. The software reuses some packages from the COIN-OR library, in particular the well-known nonlinear programming solver Ipopt, features a user-friendly interface and can be deployed on Windows / Mac / Linux.

The project is supported by Inria with the recruitment of Vincent Grelard as developer in 2010-2012, and then Daphné Giorgi since October 2012. The first prototype was released at the end of 2011, Bocop is currently at version 1.1.4 and has been downloaded more than 700 times. The software was first succesfully tested on several academic problems, see available on http://bocop.org. Starting in 2012, several research collaborations were initiated in fields such as bio-reactors for energy production (, ), swimming micro-robots (), and quantum control for medical imaging (). Bocop was also featured during our participation in the Imatch "Optimisation and Control" in october, which resulted in a contract with the startup Safety Line (aeronautics).

*Bocop auto-assessment according to Inria notice: A3up4, SO3, SM3, EM3up4, SDL4up5*

This software simulates the evolution of controlled dynamical systems (possibly under uncertainties). The numerical algorithm here is based on HJB or viability approaches, and allows the design of optimal planning strategies (according to a criterion determined by the user: time, energy, ...). It also provides conflict resolution and avoidance of collisions with fixed or moving obstacles. So far, the software is used in collaboration with DGA for avoidance collision of UaVs, and by Volkswagen in some studies related to collision avoidance of cars.

This is a software for optimisation-based controller design for operating in different regimes or modes of operation. The software can be used, for example, to determine the optimal management for hybrid vehicles or hybrid engines with multiple energy sources. However, the methods used in software are still quite general and can be used in many applications.

Web page: http://www.ensta-paristech.fr/ zidani/BiNoPe-HJ

This project aims at developping sequential and parallel MPI/openMP C++ solvers for the approximation of Hamilton-Jacobi-Bellman (HJB) equations in a d-dimensional space. The main goal is to provide an HJB solvers that can work in dimension d (limited by the machine's capacity). The solver outputs can be visualized with Matlab or Paraview (via VTK files).

The development of the HJB Solver has been initiated under a partnership between COMMANDS and the SME HPC-project in the period between December 2009 to November 2011. Currently, it is still maintained and improved by COMMANDS.

In 2012, two versions were released:

HJB-SEQUENTIAL-REF: sequential version that can run on any machine

HJB-PARALLEL-REF: parallel version that can run only on multi-core architectures.

Shoot was designed for the resolution of optimal control problems via indirect methods (necessary conditions, Pontryagin's Maximum Principle). Such methods transform the original problem into finding a zero of a certain shooting function. The package offers several choices of integrators and solvers, and can handle control discontinuities. Features also include the use of variational equations to compute the Jacobian of the shooting function, as well as homotopy and grid shooting techniques for easier initialization. Shoot is an academic software, and was used during several research contracts with the CNES (french space agency).

In the report we present new models, numerical simulations and rigorous analysis for the optimization of the velocity in a race. In a seminal paper, Keller , explained how a runner should determine his speed in order to run a given distance in the shortest time. We extend this analysis, based on the equation of motion and aerobic energy, to include a balance of anaerobic energy (or accumulated oxygen deficit) and an energy recreation term when the speed decreases. We also take into account that when the anaerobic energy gets too low, the oxygen uptake cannot be maintained to its maximal value. Using optimal control theory, we obtain a proof of Keller's optimal race, and relate the problem to a relaxed formulation, where the propulsive force represents a probability distribution rather than a value function of time. Our analysis leads us to introduce a bound on the variations of the propulsive force to obtain a more realistic model which displays oscillations of the velocity. Our numerical simulations qualitatively reproduce quite well physiological measurements on real runners. We show how, by optimizing over a period, we recover these oscillations of speed. We point out that our numerical simulations provide in particular the exact instantaneous anaerobic energy used in the exercise.

In collaboration with team McTAO (Sophia), we studied in and the contrast imaging problem in nuclear magnetic resonance, modeled as Mayer problem in optimal control. The optimal solution can be found as an extremal, solution of the Maximum Principle and analyzed with the techniques of geometric control. A first synthesis of locally optimal solutions is given in the single-input case, with some preliminary results in the bi-input case. We conducted a comprehensive numerical investigation of the problem, using a combination of indirect shooting (HamPath software) and direct method (Bocop), with a moment-based (LMI) technique to estimate the global optimum.

An optimal finite-time horizon feedback control problem for (semi linear) wave equations is studied in . The feedback law can be derived from the dynamic programming principle and requires to solve the evolutionary Hamilton-Jacobi-Bellman (HJB) equation. Classical discretization methods based on finite elements lead to approximated problems governed by ODEs in high dimensional space which makes infeasible the numerical resolution by HJB approach. In the present paper, an approximation based on spectral elements is used to discretize the wave equation. The effect of noise is considered and numerical simulations are presented to show the relevance of the approach

The works , deal
with deterministic control problems where the dynamic and the
running cost can be completely different in two (or more)
complementary domains of the space

In Laurent Pfeiffer's PhD, we study stochastic optimal control problems with a probability constraint on the final state. This constraint must be satisfied with a probability greater or equal than a given level. We analyse and compare two approaches for discrete-time problems: a first one based on a dynamic programming principle and a second one using Lagrange relaxation. These approaches can be used for continuous-time problems, for which we give numerical illustrations.

Following the “iMatch Contrôle Optimisation” event held at Inria Saclay on October 23rd (2012), a collaboration was initiated between COMMANDS and the startup Safety Line (http ://www.safety-line.fr), with a first contract on optimizing the ascent phase for commercial planes. A crucial aspect of this work is the identification of accurate and reliable models for the aerodynamic and thrust forces acting on the plane. For this study our partners at Safety Line provide us access to data recorded during several thousands of actual commercial flights, and COMMANDS recruited Stephan Maindrault as engineer to work on this project.

This contract between CNES and ENSTA lasted from February to December 2013, and was devoted to trajectory global optimization for an Ariane 5 launcher, using HJB techniques. The optimization was on the whole launch, including ballistic phases and the parameters of the intermediate GTO orbit, while maximizing the payload mass.

This project is a collaboration in the framework of a 3-year (2012-2015) research program funded by DGA. The title of the project is “Problèmes de commande optimale pour des systèmes non-linéaires en présence d’incertitudes et sous contraintes de probabilité de succès”.

The team is part of the collaborative project HJNet funded by the French National Research Agency (ANR-12-BS01-0008-01). It started in January 2013 and will end in December 2013. Website: http://hjnet.math.cnrs.fr

Instrument: Initial Training Network

Duration: January 2011 - December 2014

Coordinator: Inria

Partner: Univ. of Louvain, Univ. Bayreuth, Univ. Porto, Univ. Rome - La Sapienza, ICL, Astrium-Eads, Astos solutions, Volkswagen, Univ. Padova, Univ. Pierre et Marie Curie.

Inria contact: Hasnaa Zidani

Abstract: Optimisation-based control systems concern the determination of control strategies for complex, dynamic systems, to optimise some measures of best performance. It has the potential for application to a wide range of fields, including aerospace, chemical processing, power systems control, transportation systems and resource economics. It is of special relevance today, because optimization provides a natural framework for determining control strategies, which are energy efficient and respect environmental constraints. The multi-partner initial training network SADCO aims at: Training young researchers and future scientific leaders in the field of control theory with emphasis on two major themes sensitivity of optimal strategies to changes in the optimal control problem specification, and deterministic controller design; Advancing the theory and developing new numerical methods; Conveying fundamental scientific contributions within European industrial sectors.

Title: Optimization and control in network economics

Inria principal investigator: Frédéric Bonnans

International Partner (Institution - Laboratory - Researcher):

University of Chile (Chile) - Center for Mathematical Modeling - Joseph Frédéric Bonnans

Duration: 2012 - 2014

See also: http://www.cmm.uchile.cl/EA_OCONET

Limited resources in telecommunication, energy, gas and water supply networks, lead to multi-agent interactions that can be seen as games or economic equilibrium involving stochastic optimization and optimal control problems. Interaction occurs within a network, where decisions on what to produce, consume, trade or plan, are subject to constraints imposed by node and link capacities, risk, and uncertainty, e.g. the capacity of generators and transmission lines; capacity of pipeline in gas supply; switches and antennas in telecommunication. At the same time, nonlinear phenomena arise from price formation as a consequence of demand-supply equilibria or multi-unit auction processes in the case of energy and telecommunication. We will focus first in this project in electricity markets in which there are producers/consumers PCs, and an agent called ISO (Independent system operator) in charge of the management of the network. One major application we have in mind is the one of smart (electrical) grids, in view of the increased use of renewable energies, that is, a massive entry of wind, geothermal, solar in particular.

The team is involved in the "Energy Optimization" group of the Inria research center in Chile (CIRIC). Several visits to Chile were conducted in relation with this project.

Prof. B.S. Goh, Curtin University, Miri, Malaysia; two weeks in February.

M.S. Aronna, Rosario University, Argentina; one month (February and November).

Arthur Marly, second year student of ENS Lyon. Two months intership. Subject: optimal control of populations with state constraints. Supervisor: F. Bonnans.

F. Bonnans is Corresponding Editor of “ESAIM:COCV” (Control, Optimization and Calculus of Variations), and Associate Editor of “Applied Mathematics and Optimization”, “Optimization, Methods and Software”, and “Series on Mathematics and its Applications, Annals of The Academy of Romanian Scientists”.

F. Bonnans is chairman of the SMAI-MODE group (the optimization group of the French Applied Mathematics Society) until June 2013.

F. Bonnans: Optimal control, 7h, M2, Ensta, France.

F. Bonnans: Continuous Optimization, 18h, M2, Ecole Polytechnique and U. Paris 6, France.

F. Bonnans: Numerical analysis of partial differential equations arising in finance and stochastic control, 24h, M2, Ecole Polytechnique and U. Paris 6, France.

H. Zidani: Optimal control, 14h, M2, Ensta, France.

H. Zidani: Numerical methods for front propagation, 21h, M2, Ensta France

PhD: Xavier Dupuis, Optimal control with or witout memories. ended Nov. 2013, F. Bonnans.

PhD : Laurent Pfeiffer, Sensitivity analysis for optimal control problems; Stochastic optimal control with probability constraints. ended Nov. 2013, F. Bonnans.

PhD: Zhiping Rao, Hamilton-Jacobi equations with discontinuous coefficients. Ended Dec. 2013, H. Zidani and N. Forcadel.

PhD in progress : Imène Ben-Latifa, Optimal multiple stopping and valuation of swing options in jump models. Oct. 2010, F. Bonnans and M. Mnif (ENIT, Tunis).

PhD in progress: Athena Picarelli, First and Second Order Hamilton-Jacobi equations for State-Constrained Control Problems. Nov. 2011, O. Bokanowski and H. Zidani

PhD in progress: Cristopher Hermosilla, Feedback controls and optimal trajectories. Nov. 2011, H. Zidani.

PhD in progress: Mohamed Assellaou, Reachability analysis for stochastic controlled systems. Oct. 2011, O. Bokanowski and H.Zidani.

PhD in progress: Benjamin Heymann, Dynamic optimization with uncertainty; application to energy production. Oct. 2013, F. Bonnans.

Pierre Martinon presented an “Unithé ou Café” talk about optimal control on November 8th, entitled “Quand le mieux n'est pas l'ennemi du bien”.