Section: New Results

Advances in Methodological Tools

Participants : Eitan Altman, Konstantin Avrachenkov, Alain Jean-Marie.

Control theory

Linear programming formulations for the discounted and long-run average Markov Decision Processes have evolved along separate trajectories. In 2006, E. Altman conjectured that the linear programming formulations of these two models are, most likely, a manifestation of general properties of singularly perturbed linear programs. In [8] K. Avrachenkov in collaboration with J. Filar and A. Stillman (Flinders Univ., Australia) and V. Gaitsgory (Macquarie Univ., Australia) demonstrate that this is, indeed, the case.

A. Jean-Marie, together with E. Hyon (Univ. Paris-Ouest Nanterre La Défense), completed the analysis of optimal admission control in a single-server queue with impatience. In the presence of a server startup cost, linear holding costs for the queue and individual costs for departures due to impatience, the optimal policy is to either serve customers whenever some are present, or never serve any customer. The situation is decided by a simple criterion comparing the cost of starting the server to a combination of the other parameters. Proving the optimality of such a simple policy is more difficult than expected, and involves the propagation of properties through the dynamic programming operator of a suitably approximated sequence of problems, following methods and results of Blok, Bhulai and Spieksma.

Game theory

Uniqueness of equilibrium

E. Altman in cooperation with M. Kumar (IIT Mumbai) and R. Sundaresan (IICs) have derived in [6] a new sufficient condition for uniqueness of equilibrium which extends the Diagonal Strict Concavity condition of Rosen. They further apply the condition to various networking examples.

Hybrid games

In collaboration with V. Gaitsgory, I. Brunetti (former member of Maestro ) and E. Altman have studied in [15] a non-zero sum game in which there are two components of the state space: one is a finite (controlled) Markov chain and the other is a vector of real numbers. Only the Markov chain is controlled; the other part of the state space evolves according to some differential equations whose parameters are the state and actions of the Markov chain. The authors have shown the existence of an asymptotic stationary equilibrium. They show how to derive epsilon equilibria policies for the original problem based on policies that are asymptotically equilibria.

Finite games

In [13] K. Avrachenkov in collaboration with V.V. Singh (LRI, Univ. Paris-Sud 11, France) consider coalition formation among players in an $n$-player finite strategic game over infinite horizon. At each time a randomly formed coalition makes a joint deviation from a current action profile such that at new action profile all the players from the coalition are strictly benefited. Such deviations define a coalitional better-response (CBR) dynamics that is in general stochastic. The CBR dynamics either converges to a $K$-stable equilibrium or becomes stuck in a closed cycle. We also assume that at each time a selected coalition makes mistake in deviation with small probability that add mutations (perturbations) into CBR dynamics. We prove that all $K$-stable equilibria and all action profiles from closed cycles, that have minimum stochastic potential, are stochastically stable. Similar statement holds for strict $K$-stable equilibria. We apply the CBR dynamics to study the dynamic formation of the networks in the presence of mutations. Under the CBR dynamics all strongly stable networks and closed cycles of networks are stochastically stable.

Dynamic Games

In a collaboration with M. Tidball (INRA, France), A. Jean-Marie considered the extension of an infinite-horizon dynamic game of groundwater extension [51], due to Provencher and Burt. As usual in this kind of models, the marginal extraction cost depends on the level of the groundwater. The goal of this paper is to point out the importance of the moment where this cost is announced to the players. We consider the case where the cost is announced before the extraction is made and the case where is announced after extractions. For both cases, we also analyse the possibility of taking into account the rainfall or not. The current literature considers only the case where the cost is announced before rain and harvesting. We characterize the equilibrium in the linear-quadratic case. We compare solutions as functions of the discount factor, with the particular cases of zero discount (myopic model) and no discount (maximization of the steady state) from the economic and the environmental points of view. We show that when the level of the groundwater is small, announcing costs after harvesting and rainfall is better from the economic and environmental point of view than the case of announcing it before harvesting and rainfall.

Queueing Theory

Retrial queues

In [10] K. Avrachenkov in collaboration with E. Morozov (Karelian Institute of Applied Mathematical Research, Russia) and B. Steyaert (Gent Univ., Belgium) study multi-class retrial queueing systems with Poisson inputs, general service times, and an arbitrary numbers of servers and waiting places. A class-$i$ blocked customer joins orbit $i$ and waits in the orbit for retrial. Orbit $i$ works like a single-server $·/M/1$ queueing system with exponential retrial time regardless of the orbit size. Such retrial systems are referred to as retrial systems with constant retrial rate. Our model is motivated by several telecommunication applications, such as wireless multi-access systems, optical networks and transmission control protocols, but represents independent theoretical interest as well. Using a regenerative approach, we provide sufficient stability conditions which have a clear probabilistic interpretation. We show that the provided sufficient conditions are in fact also necessary, in the case of a single-server system without waiting space and in the case of symmetric classes. We also discuss a very interesting case, when one orbit is unstable, whereas the rest of the system is stable.

In [9] K. Avrachenkov in collaboration with E. Morozov, R. Nekrasova (Karelian Institute of Applied Mathematical Research, Russia), and B. Steyaert (Gent Univ., Belgium) study the stability of a single-server retrial queueing system with constant retrial rate, general input and service processes. First, we present a review of some relevant recent results related to the stability criteria of similar systems. Sufficient stability conditions were obtained by (Avrachenkov and Morozov, 2014), which hold for a rather general retrial system. However, only in case of Poisson input an explicit expression is provided; otherwise one has to rely on simulation. On the other hand, the stability criteria derived by (Lillo, 1996) can be easily computed, but only hold for the case of exponential service times. We present new sufficient stability conditions, which are less tight than the ones obtained by (Avrachenkov and Morozov, 2010), but have an analytical expression under rather general assumptions. A key assumption is that interarrival times belongs to the class of new better than used (NBU) distributions. We illustrate the accuracy of the condition based on this assumption (in comparison with known conditions when possible) for a number of non-exponential distributions.

Polling Systems

In [12] K. Avrachenkov in collaboration with E. Perel and U. Yechiali (Tel Aviv Univ., Israel) consider a system of two separate finite-buffer $M/M/1$ queues served by a single server, where the switching mechanism between the queues is threshold-based, determined by the queue which is not being served. Applications may be found in data centers, smart traffic-light control and human behavior. We analyse both work-conserving and non-work-conserving policies. We present occasions where the non-work-conserving policy is more economical than the work-conserving policy when high switching costs are involved. An intrinsic feature of the process is an oscillation phenomenon: when the occupancy of one queue decreases, the occupancy of the other queue increases. This fact is illustrated and discussed. By formulating the system as a three-dimensional continuous-time Markov chain we provide a probabilistic analysis of the system and investigate the effects of buffer sizes and arrival rates, as well as service rates, on the system's performance. Numerical examples are presented and extreme cases are investigated.