Section: New Results

Bayesian Optimization approaches to find Nash equilibria

Participants : Mickael Binois [Univ. of Chicago] , Victor Picheny [INRA, Toulouse] , Abderrahmane Habbal.

Our aim here is to show that the Bayesian Optimization -BO- apparatus can be applied to the search of game equilibria, and in particular the classical Nash equilibrium (NE), known to be very costly the compute, notably when involved in the framework of large scale scientific computing areas.

BO relies on Gaussian processes, which are used as emulators (or surrogates) of the black-box model outputs based on a small set of model evaluations. Posterior distributions provided by the Gaussian process are used to design acquisition functions that guide sequential search strategies that balance between exploration and exploitation.

We have proposed in [26] a novel approach to solve Nash games with drastically limited budgets of evaluations based on GP regression, taking the form of a Bayesian optimization algorithm. Experiments on challenging benchmark problems demonstrate the potential of this approach compared to classical, derivative-based algorithms.

On the test problems, two acquisition functions performed similarly well. The first one, Stepwise Uncertainty Reduction -SUR- has recently emerged in the machine learning community. We introduced a new one, the Probability of Equilibrium $P_E$, which has the benefit of not relying on conditional simulation paths, which makes it simpler to implement and less computationally intensive in most cases. Still, the SUR approach has several decisive advantages; in particular, it does not actually require the new observations to belong to the grid (sampling of the , such that it could be optimized continuously. Moreover, it lays the groundwork for many extensions that may be pursued in future work.

First, SUR strategies are well-suited to allow selecting batches of points instead of only one, a key feature in distributed computer experiments. Second, other games and equilibria may be considered: the versatility of the SUR approach may allow its transposition to other frameworks, such as mixed-strategies or Bayesian games. In particular, our framework transposes directly to the case of noisy evaluations, as it can be directly modeled by the GPs without affecting the acquisition functions.