Section: New Results

Optimal control and zero-sum games

Fixed points of order preserving homogeneous maps and zero-sum games

Participants : Marianne Akian, Stéphane Gaubert, Antoine Hochart.

The PhD work of Antoine Hochart [12] deals with the applications of methods of non-linear fixed point theory to zero-sum games.

A highlight of his PhD is the characterization of the property of ergodicity for zero-sum games. In the special “zero-player” case, i.e., for a Markov chain equipped with an additive functional (payment) of the trajectory, the ergodicity condition entails that the mean payoff is independent of the initial state, for any choice of the payment. In the case of finite Markov chains, ergodicity admits several characterizations, including a combinatorial one (the uniqueness of the final class). This carries over to the two player case: ergodicity is now characterized by the absence of certain pairs of conjugate invariant sets (dominions), and it can be checked using directed hypergraphs algorithms. This leads to an explicit combinatorial sufficient condition for the solvability of the “ergodic equation”, which is the main tool in the numerical approach of the mean payoff problem. These results appeared in  [59], [58], [60]. A more general approach was developed in [30], in which zero-sum games are now studied abstractly in terms of accretive operators. This allows one to show that the bias vector (the solution of the ergodic equation) is unique for a generic perturbation of the payments.

Another series of results of the thesis concern the finite action space, showing that the set of payments for which the bias vector is not unique coincides with the union of lower dimensional cells of a polyhedral complex, which an application to perturbation schemes in policy iteration [47].

A last result of the thesis is a representation theorem for “payment free” Shapley operators, showing that these are characterized by monotonicity and homogeneity axioms [48]. This extends to the two-player case known representation theorems for risk measures.

Probabilistic and max-plus approximation of Hamilton-Jacobi-Bellman equations

Participants : Marianne Akian, Eric Fodjo.

The PhD thesis of Eric Fodjo concerns stochastic control problems obtained in particular in the modelisation of portfolio selection with transaction costs. The dynamic programming method leads to a Hamilton-Jacobi-Bellman partial differential equation, on a space with a dimension at least equal to the number of risky assets. The curse of dimensionality does not allow one to solve numerically these equations for a large dimension (greater to 5). We propose to tackle these problems with numerical methods combining policy iterations, probabilistic discretisations, max-plus discretisations, in order to increase the possible dimension. Another solution is to replace policy iterations by an approximation with optimal switching problems.

In [27], [26] (also presented in [35], [23]), we consider fully nonlinear Hamilton-Jacobi-Bellman equations associated to diffusion control problems with finite horizon involving a finite set-valued (or switching) control and possibly a continuum-valued control. We construct a lower complexity probabilistic numerical algorithm by combining the idempotent expansion properties obtained by McEneaney, Kaise and Han  [93], [99] for solving such problems with a numerical probabilistic method such as the one proposed by Fahim, Touzi and Warin  [78] for solving some fully nonlinear parabolic partial differential equations, when the volatility does not oscillate too much. Numerical tests on a small example of pricing and hedging an option are presented. Moreover, more recently, we improved the method of Fahim, Touzi and Warin to allow one to solve fully nonlinear parabolic partial differential equations with general volatilities.