The core component of our scientific agenda focuses on the development of statistical and probabilistic methods for the modeling and the optimization of complex systems. These systems require dynamic and stochastic mathematical representations with discrete and/or continuous variables. Their complexity poses genuine scientific challenges that can be addressed through complementary approaches and methodologies:

*Modeling:* design and analysis of realistic and tractable models for such complex real-life systems taking into account various probabilistic phenomena;

*Estimation:* developing theoretical and computational methods in order to estimate the parameters of the model and to evaluate the performance of the system;

*Control:* developing theoretical and numerical control tools to optimize the performance.

These three approaches are strongly connected and the most important feature of the team is to consider these topics as a whole. This enables the team to deal with real industrial problems in several contexts such as biology, production planning, trajectory generation and tracking, performance and reliability.

The scientific objectives of the team are to provide mathematical tools for modeling and optimization of complex systems. These systems require mathematical representations which are in essence dynamic, multi-model and stochastic. This increasing complexity poses genuine scientific challenges in the domain of modeling and optimization. More precisely, our research activities are focused on stochastic optimization and (parametric, semi-parametric, multidimensional) statistics which are complementary and interlinked topics. It is essential to develop simultaneously statistical methods for the estimation and control methods for the optimization of the models.

**Stochastic modeling**: Markov chain, Piecewise Deterministic Markov Processes (PDMP), Markov Decision Processes (MDP).

The mathematical representation of complex systems is a preliminary step to our final goal corresponding to the optimization of its performance. The team CQFD focuses on two complementary types of approaches. The first approach is based on mathematical representations built upon physical models where the dynamic of the real system is described by *stochastic processes*. The second one consists in studying the modeling issue in an abstract framework where the real system is considered as black-box. In this context, the outputs of the system are related to its inputs through a *statistical model*.
Regarding stochastic processes, the team studies Piecewise Deterministic Markov Processes (PDMPs) and Markov Decision Processes (MDPs). These two classes of Markov processes form general families of controlled stochastic models suitable for the design of sequential decision-making problems. They appear in many fields such as biology, engineering, computer science, economics, operations research and provide powerful classes of processes for the modeling of complex systems. Our contribution to this topic consists in expressing real-life industrial problems into these mathematical frameworks.
Regarding statistical methods, the team works on dimension reduction models. They provide a way to understand and visualize the structure of complex data sets. Furthermore, they are important tools in several different areas such as data analysis and machine learning, and appear in many applications such as biology, genetics, environment and recommendation systems. Our contribution to this topic consists in studying semiparametric modeling which combines the advantages of parametric and nonparametric models.

**Estimation methods**: estimation for PDMP; estimation in non- and semi- parametric regression modeling.

To the best of our knowledge, there does not exist any general theory for the problems of estimating parameters of PDMPs although there already exist a large number of tools for sub-classes of PDMPs such as point processes and marked point processes. To fill the gap between these specific models and the general class of PDMPs, new theoretical and mathematical developments will be on the agenda of the whole team. In the framework of non-parametric regression or quantile regression, we focus on kernel estimators or kernel local linear estimators for complete data or censored data. New strategies for estimating semi-parametric models via recursive estimation procedures have also received an increasing interest recently. The advantage of the recursive estimation approach is to take into account the successive arrivals of the information and to refine, step after step, the implemented estimation algorithms. These recursive methods do require restarting calculation of parameter estimation from scratch when new data are added to the base. The idea is to use only the previous estimations and the new data to refresh the estimation. The gain in time could be very interesting and there are many applications of such approaches.

**Dimension reduction**: dimension-reduction via SIR and related methods, dimension-reduction via multidimensional and classification methods.

Most of the dimension reduction approaches seek for lower dimensional subspaces minimizing the loss of some statistical information. This can be achieved in modeling framework or in exploratory data analysis context.

In modeling framework we focus our attention on semi-parametric models in order to conjugate the advantages of parametric and nonparametric modeling. On the one hand, the parametric part of the model allows a suitable interpretation for the user. On the other hand, the functional part of the model offers a lot of flexibility.
In this project, we are especially interested in the semi-parametric regression model

Methods of dimension reduction are also important tools in the field of data analysis, data mining and machine learning.They provide a way to understand and visualize the structure of complex data sets.Traditional methods among others are principal component analysis for quantitative variables or multiple component analysis for qualitative variables. New techniques have also been proposed to address these challenging tasks involving many irrelevant and redundant variables and often comparably few observation units. In this context, we focus on the problem of synthetic variables construction, whose goals include increasing the predictor performance and building more compact variables subsets. Clustering of variables is used for feature construction. The idea is to replace a group of ”similar” variables by a cluster centroid, which becomes a feature. The most popular algorithms include K-means and hierarchical clustering. For a review, see, e.g., the textbook of Duda .

**Stochastic control**: optimal stopping, impulse control, continuous control, linear programming.

The main objective is to develop *approximation techniques* to provide quasi-optimal feasible solutions and to derive *optimality results* for control problems related to MDPs and PDMPs:

*Approximation techniques*.
The analysis and the resolution of such decision models mainly rely on the maximum principle and/or the dynamic/linear programming techniques together with their various extensions such as the value iteration (VIA) and the policy iteration (PIA) algorithm. However, it is well known that these approaches are hardly applicable in practice and suffer from the so-called *curse of dimensionality*. Hence, solving numerically a PDMP or an MDP is a difficult and important challenge.
Our goal is to obtain results which are both consistent from a theoretical point of view and computationally tractable and accurate from an application standpoint.
It is important to emphasize that these research objectives were not planned in our initial 2009 program.

Our objective is to propose approximation techniques to efficiently compute the optimal value function and to get quasi-optimal controls for different classes of constrained and unconstrained MDPs with general state/action spaces, and possibly unbounded cost function. Our approach is based on combining the linear programming formulation of an MDP with probabilistic approximation techniques related to quantization techniques and the theory of empirical processes. An other aim is to apply our methods to specific industrial applications in collaboration with industrial partners such as Airbus Defence & Space, DCNS and Thales.

Asymptotic approximations are also developed in the context of queueing networks, a class of models where the decision policy of the underlying MDP is in some sense fixed a priori, and our main goal is to study the transient or stationary behavior of the induced Markov process. Even though the decision policy is fixed, these models usually remain intractable to solve. Given this complexity, the team has developed analyses in some limiting regime of practical interest, i.e., queueing models in the large-network, heavy-traffic, fluid or mean-field limit. This approach is helpful to obtain a simpler mathematical description of the system under investigation, which is often given in terms of ordinary differential equations or convex optimization problems.

*Optimality results*.
Our aim is to investigate new important classes of optimal stochastic control problems including constraints and combining continuous and impulse actions for MDPs and PDMPs. In this framework, our objective is to obtain different types of optimality results. For example, we intend to provide conditions to guarantee the existence and uniqueness of the optimality equation for the problem under consideration and to ensure existence of an optimal (and

Our abilities in probability and statistics apply naturally to industry, in particular in studies of dependability and safety. An illustrative example is the collaboration that started in September 2014 with with THALES Optronique. The goal of this project is the optimization of the maintenance of an onboard system equipped with a HUMS (Health Unit Monitoring Systems). The physical system under consideration is modeled by a piecewise deterministic Markov process. In the context of impulse control, we propose a dynamic maintenance policy, adapted to the state of the system and taking into account both random failures and those related to the degradation phenomenon.

The spectrum of applications of the topics that the team can address is large and can concern many other fields. Indeed non parametric and semi-parametric regression methods can be used in biometry, econometrics or engineering for instance. Gene selection from microarray data and text categorization are two typical application domains of dimension reduction among others. We had for instance the opportunity via the scientific program PRIMEQUAL to work on air quality data and to use dimension reduction techniques as principal component analysis (PCA) or positive matrix factorization (PMF) for pollution sources identification and quantization.

Pierre Del Moral is a Simons foundation CRM Professor, Montréal Math. Research Center 2017

Google scholar classic paper in Probability and Statistics (ten most-cited articles
published ten years earlier): Del Moral, P., Doucet A., Jasra A.. *Sequential Monte Carlo Samplers* Journal of the Royal Statistical Society, Series B, vol. 68, no. 3, pp. 411-436 (2006).

*Bayesian Inference with Interacting Particle Systems*

Functional Description: Biips is a software platform for automatic Bayesian inference with interacting particle systems. Biips allows users to define their statistical model in the probabilistic programming BUGS language, as well as to add custom functions or samplers within this language. Then it runs sequential Monte Carlo based algorithms (particle filters, particle independent Metropolis-Hastings, particle marginal Metropolis-Hastings) in a black-box manner so that to approximate the posterior distribution of interest as well as the marginal likelihood. The software is developed in C++ with interfaces with the softwares R, Matlab and Octave.

Participants: Adrien Todeschini and François Caron

Contact: Adrien Todeschini

Keyword: Statistic analysis

Functional Description: Mixed data type arise when observations are described by a mixture of numerical and categorical variables. The R package PCAmixdata extends standard multivariate analysis methods to incorporate this type of data. The key techniques included in the package are PCAmix (PCA of a mixture of numerical and categorical variables), PCArot (rotation in PCAmix) and MFAmix (multiple factor analysis with mixed data within a dataset). The MFAmix procedure handles a mixture of numerical and categorical variables within a group - something which was not possible in the standard MFA procedure. We also included techniques to project new observations onto the principal components of the three methods in the new version of the package.

Contact: Marie Chavent

URL: https://

Keyword: Regression

Functional Description: QuantifQuantile is an R package that allows to perform quantization-based quantile regression. The different functions of the package allow the user to construct an optimal grid of N quantizers and to estimate conditional quantiles. This estimation requires a data driven selection of the size N of the grid that is implemented in the functions. Illustration of the selection of N is available, and graphical output of the resulting estimated curves or surfaces (depending on the dimension of the covariate) is directly provided via the plot function.

Contact: Jérôme Saracco

In many distributed computing systems, stochastically arriving jobs need to be assigned to servers with the objective of minimizing waiting times. Many existing dispatching algorithms are basically included in the SQ

Author: J. Anselmi (Inria CQFD).

Authors: J. Anselmi (Inria CQFD), D. Ardagna, J.C.S. Lui, A. Wierman, Y. Xu and Z. Yang.

Piecewise-deterministic Markov processes form a general class of non-diffusion stochastic models that involve both deterministic trajectories and random jumps at random times. In , we state a new characterization of the jump rate of such a process with discrete transitions. We deduce from this result a nonparametric technique for estimating this feature of interest. We state the uniform convergence in probability of the estimator. The methodology is illustrated on a numerical example.

Authors: A. Genadot (Inria CQFD) and R. Azais.

Authors: E. Costa and B. De Saporta (Inria CQFD).

Authors: O. Costa and F. Dufour (Inria CQFD).

Authors: B. De Saporta, F. Dufour and A. Geeraert. All authors are members of CQFD at Inria.

Authors: B. De Saporta, F. Dufour and C. Nivot. All authors are members of CQFD at Inria.

Authors: P. Del Moral (Inria CQFD), A. Kurtzmann and J. Tugaut.

Authors: P. Del Moral (Inria CQFD) and D. Villemonais.

Authors: A. Genadot (Inria CQFD).

Sequential and quantum Monte Carlo methods, as well as genetic type search algorithms can be interpreted as a mean field and interacting particle approximations of Feynman-Kac models in distribution spaces. The performance of these population Monte Carlo algorithms is strongly related to the stability properties of nonlinear Feynman–Kac semigroups. In , we analyze these models in terms of Dobrushin ergodic coefficients of the reference Markov transitions and the oscillations of the potential functions. Sufficient conditions for uniform concentration inequalities w.r.t. time are expressed explicitly in terms of these two quantities. We provide an original perturbation analysis that applies to annealed and adaptive Feynman-Kac models, yielding what seems to be the first results of this kind for these types of models. Special attention is devoted to the particular case of Boltzmann-Gibbs measures' sampling. In this context, we design an explicit way of tuning the number of Markov chain Monte Carlo iterations with temperature schedule. We also design an alternative interacting particle method based on an adaptive strategy to define the temperature increments. The theoretical analysis of the performance of this adaptive model is much more involved as both the potential functions and the reference Markov transitions now depend on the random evolution on the particle model. The nonasymptotic analysis of these complex adaptive models is an open research problem. We initiate this study with the concentration analysis of a simplified adaptive models based on reference Markov transitions that coincide with the limiting quantities, as the number of particles tends to infinity.

Authors: F. Giraud and P. Del Moral (Inria CQFD).

Genetic programming (GP) is an evolutionary computation paradigm for automatic program induction. GP has produced impressive results but it still needs to overcome some practical limitations, particularly its high computational cost, overfitting and excessive code growth. Recently, many researchers have proposed fitness-case sampling methods to overcome some of these problems, with mixed results in several limited tests. In , we present an extensive comparative study of four fitness-case sampling methods, namely: Interleaved Sampling, Random Interleaved Sampling, Lexicase Selection and Keep-Worst Interleaved Sampling. The algorithms are compared on 11 symbolic regression problems and 11 supervised classification problems, using 10 synthetic benchmarks and 12 real-world data-sets. They are evaluated based on test performance, overfitting and average program size, comparing them with a standard GP search. Comparisons are carried out using non-parametric multigroup tests and post hoc pairwise statistical tests. The experimental results suggest that fitness-case sampling methods are particularly useful for difficult real-world symbolic regression problems, improving performance, reducing overfitting and limiting code growth. On the other hand, it seems that fitness-case sampling cannot improve upon GP performance when considering supervised binary classification.

Authors: Y. Martinez, E. Naredo, L. Trujillo, P. Legrand (Inria CQFD) and U. Lopez.

Authors: H. Zhang (Inria CQFD), B. De Saporta (Inria CQFD), F. Dufour (Inria CQFD), D. Laneuville and A. Nègre.

There are two important limitations of standard tree-based genetic programming (GP). First, GP tends to evolve unnecessarily large programs, what is referred to as bloat. Second, GP uses inefficient search operators that focus on modifying program syntax. The first problem has been studied in many works, with many bloat control proposals. Regarding the second problem, one approach is to use alternative search operators, for instance geometric semantic operators, to improve convergence. In , our goal is to experimentally show that both problems can be effectively addressed by incorporating a local search optimizer as an additional search operator. Using real-world problems, we show that this rather simple strategy can improve the convergence and performance of tree-based GP, while reducing program size. Given these results, a question arises: why are local search strategies so uncommon in GP? A small survey of popular GP libraries suggests to us that local search is underused in GP systems.

Authors: Leonardo Trujillo, Emigdio Z-Flores, Perla S. Juarez Smith, Pierrick Legrand (Inria CQFD), Sara Silva, Mauro Castelli, Leonardo Vanneschi, Oliver Schutze and Luis Munoz.

Authors: Marie Chavent (Inria CQFD), Vanessa Kuentz, Amaury Labenne, Jérôme Saracco (Inria CQFD).

Authors: Ines Jlassi, Jérôme Saracco (Inria CQFD).

Authors: Guy Chavent, Marie Chavent (Inria CQFD).

Authors: Marie Chavent (Inria CQFD), Vanessa Kuentz, Amaury Labenne, Jérôme Saracco (Inria CQFD).

Authors: Ines Jlassi, Jérôme Saracco (Inria CQFD).

Authors: Pierre Del Moral (Inria CQFD), Adrian Bishop and Sahani Pathiraja.

Collision between satellites and space debris seldom happens, but the loss of a satellite by collision may have catastrophic consequences both for the satellite mission and for the space environment. To support the decision to trigger o a collision avoidance manoeuver, an adapted tool is the determination of the collision probability between debris and satellite. This probability estimation can be performed with rare event simulation techniques when Monte Carlo techniques are not enough accurate. In this chapter, we focus on analyzing the inuence of dierent simulation parameters (such as the drag coecient) that are set for to simplify the simulation, on the collision probability estimation. A bad estimation of these simulation parameters can strongly modify rare event probability estimations. We design here a new island particle Markov chain Monte Carlo algorithm to determine the parameters that, in case of bad estimation, tend to increase the collision probability value. This algorithm also gives an estimate of the collision probability maximum taking into account the likelihood of the parameters. The principles of this statistical technique are described throughout this chapter.

Authors: Pierre Del Moral (Inria CQFD), Christelle Vergé, Jérôme Morio and Juan Carlos Dolado Pérez.

We consider Bayesian online static parameter estimation for state-space models. This is a very important problem, but is very computationally challenging as the state-of-the art methods that are exact, often have a computational cost that grows with the time parameter; perhaps the most successful algorithm is that of SM C2 (Chopin et al., J R Stat Soc B 75: 397–426 2013). We present a version of the SM C2 algorithm which has computational cost that does not grow with the time parameter. In addition, under assumptions, the algorithm is shown to provide consistent estimates of expectations w.r.t. the posterior. However, the cost to achieve this consistency can be exponential in the dimension of the parameter space; if this exponential cost is avoided, typically the algorithm is biased. The bias is investigated from a theoretical perspective and, under assumptions, we find that the bias does not accumulate as the time parameter grows. The algorithm is implemented on several Bayesian statistical models.

Authors: Pierre Del Moral (Inria CQFD), Ajay Jasra and Yan Zhou.

This work considers the sequential Monte Carlo (SMC) approximation of ratios of normalizing constants associated to posterior distributions which in principle rely on continuum models. Therefore, the Monte Carlo estimation error and the discrete approximation error must be balanced. A multilevel strategy is utilized to substantially reduce the cost to obtain a given error level in the approximation as compared to standard estimators. Two estimators are considered and relative variance bounds are given. The theoretical results are numerically illustrated for the example of identifying a parametrized permeability in an elliptic equation given point-wise observations of the pressure.

Authors: Pierre Del Moral (Inria CQFD), Ajay Jasra, Kody Law and Yan Zhou.

In this article, we consider the multilevel sequential Monte Carlo (MLSMC) method of Beskos et al. (Stoch. Proc. Appl. [to appear]). This is a technique designed to approximate expectations w.r.t. probability laws associated to a discretization. For instance, in the context of inverse problems, where one discretizes the solution of a partial differential equation. The MLSMC approach is especially useful when independent, coupled sampling is not possible. Beskos et al. show that for MLSMC the computational effort to achieve a given error, can be less than independent sampling. In this article we significantly weaken the assumptions of Beskos et al., extending the proofs to non-compact state-spaces. The assumptions are based upon multiplicative drift conditions as in Kontoyiannis and Meyn (Electron. J. Probab. 10 [2005]: 61–123). The assumptions are verified for an example.

Authors: Pierre Del Moral (Inria CQFD), Ajay Jasra and Kody Law.

We consider Bayesian online static parameter estimation for state-space models. This is a very important problem, but is very computationally challenging as the state-of-the art methods that are exact, often have a computational cost that grows with the time parameter; perhaps the most successful algorithm is that of SM C2 (Chopin et al., J R Stat Soc B 75: 397–426 2013). We present a version of the SM C2 algorithm which has computational cost that does not grow with the time parameter. In addition, under assumptions, the algorithm is shown to provide consistent estimates of expectations w.r.t. the posterior. However, the cost to achieve this consistency can be exponential in the dimension of the parameter space; if this exponential cost is avoided, typically the algorithm is biased. The bias is investigated from a theoretical perspective and, under assumptions, we find that the bias does not accumulate as the time parameter grows. The algorithm is implemented on several Bayesian statistical models.

Authors: Pierre Del Moral (Inria CQFD), Ajay Jasra and Yan Zhou.

Sequential Monte Carlo (SMC) methods have successfully been used in many applications in engineering, statistics and physics. However, these are seldom used in financial option pricing literature and practice. This paper presents SMC method for pricing barrier options with continuous and discrete monitoring of the barrier condition. Under the SMC method, simulated asset values rejected due to barrier condition are re-sampled from asset samples that do not breach the barrier condition improving the efficiency of the option price estimator; while under the standard Monte Carlo many simulated asset paths can be rejected by the barrier condition making it harder to estimate option price accurately. We compare SMC with the standard Monte Carlo method and demonstrate that the extra effort to implement SMC when compared with the standard Monte Carlo is very little while improvement in price estimate can be significant. Both methods result in unbiased estimators for the price converging to the true value as

Authors: Pierre Del Moral (Inria CQFD) and Pavel V. Shevchenko.

The increasing complexity of warfare submarine missions has led DCNS to study new tactical help functions for underwater combat management systems. In this context, the objective is to find optimal trajectories according to the current mission type by taking into account sensors, environment and surrounding targets. This problem has been modeled as a discrete-time Markov decision process with finite horizon. A quantization technique has been applied to discretize the problem in order to get a finite MDP for which standard methods such as the dynamic and/or the linear programming approaches can be applied. Different kind of scenarios have been considered and studied.

Maintenance, impulse control, failure, optimization. The objective of this grant in collaboration with Thales Optronique was to optimize the maintenance of a multi-component equipment that can break down randomly. The underlying problem was to choose the best dates to repair or replace components in order to minimize a cost criterion that takes into account costs of maintenance but also the cost associated to the unavailability of the system for the customer. This industrial process has been modeled by a piecewise deterministic Markov process (PDMP) and the maintenance problem has been formalized as an impulse control problem. We have applied an approximation method based on a quantization technique of the post jump location and inter-arrival time Markov chain naturally embedded in the PDMP, and a path-adapted time discretization grids to get an approximation of the value function. We have shown the existence of control strategies that can outperform reference control policies used by Thales Optronique. It remains to provide the explicit form of such strategies. This is actually the objective of a new collaboration with Thales Optronique that started in October 2017 funded by the Fondation Mathématique Jacques Hadamard.

Stochastic modelling, Optimization. This project has just started in November 2017. The topic of this collaboration with Lyre, l'Agence de l'eau Adour-Garonne and ENSEGID is the modeling of the uncertainties in the Water demand adequacy in a context of global climate change. A PhD thesis (2018-2021) is part of this project.

Piecewise Deterministic Markov Processes (PDMP) are non-diffusive stochastic processes which naturally appear in many areas of applications as communication networks, neuron activities, biological populations or reliability ofcomplex systems. Their mathematical study has been intensively carried out in the past two decades but many challenging problems remain completely open. This project aims at federating a group of experts with different backgrounds (probability, statistics, analysis, partial derivative equations, modelling) in order to pool everyone's knowledge and create new tools to study PDMPs. The main lines of the project relate to estimation, simulation and asymptotic behaviors (long time, large populations, multi-scale problems) in the various contexts of application.

The involved research groups are Inria Rennes/IRISA Team SUMO; Inria Rocquencourt Team Lifeware; LIAFA University Paris 7; Bordeaux University.

The aim of this research project is to develop scalable model checking techniques that can handle large stochastic systems. Large stochastic systems arise naturally in many different contexts, from network systems to system biology. A key stochastic model we will consider is from the biological pathway of apoptosis, the programmed cell death.

Statistical methods have become more and more popular in signal and image processing over the past decades. These methods have been able to tackle various applications such as speech recognition, object tracking, image segmentation or restoration, classification, clustering, etc. We propose here to investigate the use of Bayesian nonparametric methods in statistical signal and image processing. Similarly to Bayesian parametric methods, this set of methods is concerned with the elicitation of prior and computation of posterior distributions, but now on infinite-dimensional parameter spaces. Although these methods have become very popular in statistics and machine learning over the last 15 years, their potential is largely underexploited in signal and image processing. The aim of the overall project, which gathers researchers in applied probabilities, statistics, machine learning and signal and image processing, is to develop a new framework for the statistical signal and image processing communities. Based on results from statistics and machine learning we aim at defining new models, methods and algorithms for statistical signal and image processing. Applications to hyperspectral image analysis, image segmentation, GPS localization, image restoration or space-time tomographic reconstruction will allow various concrete illustrations of the theoretical advances and validation on real data coming from realistic contexts.

The involved research groups are Inria Bordeaux Sud-Ouest Team CQFD and Thales Optronique. This new collaboration with Thales Optronique that started in October 2017 is funded by the Fondation Mathématique Jacques Hadamard. This is the continuation of the PhD Thesis of A. Geeraert. The objective of this project is to optimize the maintenance of a multi-component equipment that can break down randomly. The underlying problem is to choose the best dates to repair or replace components in order to minimize a cost criterion that takes into account costs of maintenance but also the cost associated to the unavailability of the system for the customer. In the PhD thesis of A. Geeraert, the model under consideration was rather simple and only a numerical approximation of the value function was provided. Here, our objective is more ambitious. A more realistic model will be considered and our aim is to provide a tractable quasi-optimal control strategy that can be applied in practice to optimize the maintenance of such equipments.

Program: Direcion General de Investigacion Cientifica y Tecnica, Gobierno de Espana

Project acronym: GAMECONAPX

Project title: Numerical approximations for Markov decision processes and Markov games

Duration: 01/2017 - 12/2019

Coordinator: Tomas Prieto-Rumeau, Department of Statistics and Operations Research, UNED (Spain)

Abstract:

This project is funded by the Gobierno de Espana, Direcion General de Investigacion Cientifica y Tecnica (reference number: MTM2016-75497-P) for three years to support the scientific collaboration between Tomas Prieto-Rumeau, Jonatha Anselmi and Francois Dufour. This research project is concerned with numerical approximations for Markov decision processes and Markov games. Our goal is to propose techniques allowing to approximate numerically the optimal value function and the optimal strategies of such problems. Although such decision models have been widely studied theoretically and, in general, it is well known how to characterize their optimal value function and their optimal strategies, the explicit calculation of these optimal solutions is not possible except for a few particular cases. This shows the need for numerical procedures to estimate or to approximate the optimal solutions of Markov decision processes and Markov games, so that the decision maker can really have at hand some approximation of his optimal strategies and his optimal value function. This project will explore areas of research that have been, so far, very little investigated. In this sense, we expect our techniques to be a breakthrough in the field of numerical methods for continuous-time Markov decision processes, but particularly in the area of numerical methods for Markov game models. Our techniques herein will cover a wide range of models, including discrete- and continuous-time models, problems with unbounded cost and transition rates, even allowing for discontinuities of these rate functions. Our research results will combine, on one hand, mathematical rigor (with the application of advanced tools from probability and measure theory) and, on the other hand, computational efficiency (providing accurate and ?applicable? numerical methods). In this sense, particular attention will be paid to models of practical interest, including population dynamics, queueing systems, or birth-and-death processes, among others. So, we expect to develop a generic and robust methodology in which, by suitably specifying the data of the decision problem, an algorithm will provide the approximations of the value function and the optimal strategies. Therefore, the results that we intend to obtain in this research project will be of interest for researchers in the fields of Markov decision processes and Markov games, both for the theoretical and the applied or practitioners communities

**Tree-Lab, ITT**. TREE-LAB is part of the Cybernetics research line within the Engineering Science graduate program offered by the Department of Electric and Electronic Engineering at Tijuana's Institute of Technology (ITT), in Tijuana Mexico. TREE-LAB is mainly focused on scientific and engineering research within the intersection of broad scientific fields, particularly Computer Science, Heuristic Optimization and Pattern Analysis. In particular, specific domains studied at TREE-LAB include Genetic Programming, Classification, Feature Based Recognition, Bio-Medical signal analysis and Behavior-Based Robotics. Currently, TREE-LAB incorporates the collaboration of several top researchers, as well as the participation of graduate (doctoral and masters) and undergraduate students, from ITT. Moreover, TREE-LAB is actively collaborating with top researchers from around the world, including Mexico, France, Spain, Portugal and USA.

Tomas Prieto-Rumeau (Department of Statistics and Operations Research, UNED, Madrid, Spain) visited the team during 2 weeks in 2017. The main subject of the collaboration is the approximation of Markov Decision Processes.

Oswaldo Costa (Escola Politécnica da Universidade de São Paulo, Brazil) collaborate with the team on the theoretical aspects of continuous control of piecewise-deterministic Markov processes. He visited the team during two weeks in 2017.

P. Legrand was co-organizer of EA 2017.

F. Dufour has been a member of the Organizing Committee of the SIAM Conference on Control and Its Applications (CT17) in Pittsburgh, USA, 2017.

P. Legrand was chair for EA 2017.

J. Anselmi has been a member of the TPC of the following international conferences: VALUETOOLS-2017, ASMTA-2017 and IFIP Performance 2017.

P. Legrand has been a member of the PC of the following international conferences: EA 2017.

P. Del Moral is an associate editor for the journal Stochastic Analysis and Applications since 2001.

P. Del Moral is an associate editor for the journal Revista de Matematica: Teoria y aplicaciones since 2009.

P. Del Moral is an associate editor for the journal Applied Mathematics and Optimization since 2009.

F. Dufour is associate editor of the journal: SIAM Journal of Control and Optimization since 2009.

F. Dufour is the representative of the SIAM activity group in control and system theory for the journal SIAM News since 2014.

J. Saracco is an associate editor of the journal Case Studies in Business, Industry and Government Statistics (CSBIGS) since 2006.

All the members of CQFD are regular reviewers for several international journals and conferences in applied probability, statistics and operations research.

Pierrick Legrand was invited to give a talk on IA in Thales center in october.

Pierre del Moral gave several invited talks listed at the following address: http://

P. Legrand was the scientific leader of the programs HUMO 3 (with UBX and IMS) and MICRO-DOPPLER (with Thales and IMS) in the context of the GIS ALBATROS.

Pierrick Legrand has been reviewer for the ANR generics projects in 2017.

J. Saracco is elected member of the council of the *Société Française de Statistique* (SFdS, French Statistical Society).

J. Saracco was vice president of SFdS from 2014 to 2016.

J. Saracco is deputy director of IMB (Institut de Mathématiques de Bordeaux, UMR CNRS 5251) since 2015.

M. Chavent is member of the national evaluation committee of Inria.

M. Chavent and Pierrick Legrand are members of the council of the Institut de Mathématique de Bordeaux.

Pierrick Legrand was the director of the Ressources Center Victoire of the UF Mathematics and interactions until may 2017.

Licence : J. Anselmi, Probabilités et statistiques, 20 heures, L3, Institut Polytechnique de Bordeaux, école ENSEIRB-MATMECA, filiÃšre Télécommunications, France.

Licence : J. Anselmi, Probabilités et statistiques, 16 heures, L3, Institut Polytechnique de Bordeaux, école ENSEIRB-MATMECA, filiÃšre Electronique, France.

Licence : J. Anselmi, Probabilités et statistiques, 48 heures, niveau L3, Institut Polytechnique de Bordeaux, école ENSEIRB-MATMECA, filiÃšre Mathématique et Mécanique, France.

Licence: M. Chavent, Analyse des données, 15 ETD, L3, Bordeaux university, France

License: M. Chavent, Modélisation statistique, 15 ETD, niveau L3, Bordeaux university, France

Master : M. Chavent, Apprentissage automatique, 50 ETD, niveau M2, Bordeaux university, France

Licence : F. Dufour, Probabilités et statistiques, 70h, first year of école ENSEIRB-MATMECA, Institut Polytechnique de Bordeaux, France.

Master : F. Dufour, Méthodes numériques pour la fiabilité, 36h, third year of école ENSEIRB-MATMECA, Institut Polytechnique de Bordeaux, France.

P. Legrand, AlgÃšbre (responsable de l'UE), Licence 1 SCIMS (108 heures)

P. Legrand, Informatique pour les mathÃÂ©matiques (responsable de l'UE), Licence 1 et Licence 2 (36 heures)

P. Legrand, Espaces Euclidiens. (responsable de l'UE), Licence 2 SCIMS (54 heures)

Licence: J. Saracco, Probability and Descriptive statistics, 27h, L3, First year of ENSC - Bordeaux INP, France

Licence: J. Saracco, Mathematical statistics, 20h, L3, First year of ENSC - Bordeaux INP, France

Licence: J. Saracco, Data analysis (multidimensional statistics), 20h, L3, First year of ENSC - Bordeaux INP, France

Master: J. Saracco, Statistical modeling, 27h, M1, Second year of ENSC - Bordeaux INP, France

Master: J. Saracco, Applied probability and Statistics, 40h, M1, Second year of ENSCBP - Bordeaux INP, France

Master: J. Saracco, Probability and Statistics, 12h, M2, Science Po Bordeaux, France

A. Genadot, Probabilités de bases (18h), Licence MIASHS premiÃšre année, Université de Bordeaux.

A. Genadot, Statistiques de bases (18h), Licence MIASHS premiÃšre année, Université de Bordeaux.

A. Genadot, Probabilités (36h), Licence MIASHS deuxiÃšme année, Université de Bordeaux.

A. Genadot, Processus (18h), Licence MIASHS troisiÃšme année, Université de Bordeaux.

A. Genadot, Modélisation statistique (18h), Licence MIASHS troisiÃšme année, Université de Bordeaux.

A. Genadot, Martingales (25h), Master MIMSE premiÃšre année, Université de Bordeaux.

A. Genadot, Probabilités (20h), Master MEEF premiÃšre année, Université de Bordeaux.

PhD completed : Alizé Geeraert, Contrôle optimal des processus Markoviens déterministes par morceaux et application à la maintenance, University of Bordeaux, supervised by B. de Saporta and F. Dufour (defense in June 2017).

PhD in progress : Ines Jlassi, Contributions à la régression inverse par tranches et à l'estimation non para métrique des quantiles conditionnels, University of Monastir (Tunisia), September 2013, supervised by J. Saracco and L. Ben Abdelghani Bouraoui.

PhD in progress : Hadrien Lorenzo, Analyses de données longitudinales de grandes dimensions appliquées aux essais vaccinaux contre le VIH et Ebola, University of Bordeaux, September 2016, supervised by J. Saracco and R. Thiebaut.

PhD in progress : Tiffany Cherchi, “Automated optimal fleet management policy for airborne equipment”, Montpellier University, since 2017, supervised by B. De Saporta and F. Dufour.

PhD in progress : Chloé Pasin, “Modelisation et optimisation de la réponse vaccinale. Application au VIH et Ebola”, Bordeaux University, since 2015, supervised by F. Dufour and R. Thiebaut.

PhD in progress : Maud Joubaud, “Branching piecewise deterministic Markov processes, applications to cell biology”, Montpellier University, since 2016, supervised by B. De Saporta and B. Cloez.

PhD completed : Emigdio Z. Flores, Human mental states classification using EEG by means of Genetic Programming, ITT Tijuana, supervised by L. Trujillo and P. Legrand (defense in July 2017).

J. Saracco is vice president of the french statistical society (SFdS).