Section: New Results

Stochastic Modeling

Participants : Eitan Altman, Konstantin Avrachenkov, Mandar Datar, Swapnil Dhamal, Alain Jean-Marie, Albert Sunny.

Markov chains with restart/jumps

In [7], K. Avrachenkov together with A. Piunovskiy and Y. Zhang (Univ. of Liverpool, UK) consider a discrete-time Markov process with restart. At each step the process either with a positive probability restarts from a given distribution, or with the complementary probability continues according to a Markov transition kernel. The main contribution of this work is an explicit expression for the expectation of the hitting time (to a given target set) of the process with restart. The formula is convenient when considering the problem of optimization of the expected hitting time with respect to the restart probability. The results with are illustrated with two examples in uncountable and countable state spaces and with an application to network centrality.

Then, in [19], K. Avrachenkov and I. Bogdanov (HSE, Russia) study the relaxation time in the random walk with jumps. The random walk with jumps combines random walk based sampling with uniform node sampling and improves the performance of network analysis and learning tasks. They derive various conditions under which the relaxation time decreases with the introduction of jumps.

Markov modeling of Lasers

A. Jean-Marie has continued the investigation of Markov models of Lasers at several levels of physical accuracy, in conjunction with F. Philippe, L. Chusseau and A. Vallet (Univ. Montpellier and CNRS). In [17], a Markov model of relatively low complexity, the “Canonical Markov Model” (CMM), is built on the basis of a time-scale decomposition of physical phenomena. This simplified model is validated by comparison with a “microscopic Markov model” previously existing. Thanks to its smaller state space, simulations with the CMM are orders of magnitude faster, and numerical investigation of stationary and transient features become possible. As an example, the focus is put in [17], [39] on the Laser “threshold”, a phenomenon related to sojourn of the CMM in states where no light is emitted. Simulations and numerical solutions reveal the existence of a bi-modal distribution for the particles for a certain range of parameters, thereby predicting a certain instability of the Laser for these values. Investigations continue with a quantification of the intensity of “flashes” through the computation of hitting times in the CMM.

The marmoteCore platform

The development of marmoteCore (see Section 6.1) has been pursued by A. Jean-Marie. The software library is now being used in Neo 's research projects such as [17] or queuing models supporting the analysis of Green Data Centers. marmoteCore provides the classes necessary to represent the state space of Markov models, from the elementary bricks that are interval or rectangular domains, simplices, or binary sequences. From there, the user easily programs the construction of probability transition matrices or infinitesimal generators. Structural analysis methods allow to identify recurrent and transient classes, and to compute the period of the model. Numerous methods allow the Monte Carlo simulation of the chain, the computation of transient and stationary distributions, as well as hitting times. In conjunction with E. Hyon (Univ. Paris-Nanterre), extensions of the core of the software are being programmed for Markov Decision Processes and Stochastic Games.

Blockchain mining

S. Dhamal, T. Chahed (Telecom SudParis), W. Ben-Ameur (Telecom SudParis), E. Altman, A. Sunny, and S. Poojary (UAPV, the Univ. of Avignon) have studied a stochastic game framework for distributed computing settings such as blockchain mining in [42]. A continuous-time Markov chain model, where players arrive and depart according to a stochastic process, is proposed, and their investment strategies are determined based on the state of the system. Two scenarios are analyzed, based on whether the rate of problem getting solved is dependent on or independent of the computational power invested by the players. The equilibrium strategies are shown to follow a threshold policy when this rate is proportional to the total invested power, while the players are shown to invest proportionally to the reward-cost ratio when this rate is independent of the invested power. The effects of arrival and departure rates on the players' utilities are quantified using simulations.

The paper extends the game theoretic modeling and analysis of the static case (fixed number of miners) done in [18] by E. Altman in collaboration with A. Reiffers-Masson (IISc, India), D. Sadoc Menasché (UFRJ), M. Datar and S. Dhamal, and C. Touati (Inria Grenoble Rhône-Alpes).