Homepage Inria website
  • Inria login
  • The Inria's Research Teams produce an annual Activity Report presenting their activities and their results of the year. These reports include the team members, the scientific program, the software developed by the team and the new results of the year. The report also describes the grants, contracts and the activities of dissemination and teaching. Finally, the report gives the list of publications of the year.

  • Legal notice
  • Cookie management
  • Personal data
  • Cookies

Section: New Results

Ergodic theory for controlled Markov chains with stationary inputs

Consider a stochastic process 𝐗 on a finite state space X={1,,d}. It is conditionally Markov, given a real-valued `input process' ζ. This is assumed to be small, which is modeled through the scaling, ζt=εζt1,0ε1, where ζ1 is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on ζ:

  • A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain 𝐗 obtained with ζ0. The triple (𝐗,𝐗,ζ) is a jointly stationary process satisfying 𝖯{X(t)X(t)}=O(ε). Moreover, a second-order Taylor-series approximation is obtained:

    𝖯 { X ( t ) = i } = 𝖯 { X ( t ) = i } + ε 2 ϱ ( i ) + o ( ε 2 ) , 1 i d ,

    with an explicit formula for the vector ϱd.

  • For any m1 and any function f:{1,,d}×m, the stationary stochastic process Y(t)=f(X(t),ζ(t)) has a power spectral density Sf that admits a second order Taylor series expansion: A function S(2)f:[-π,π]Cm×m is constructed such that

    S f ( θ ) = S f ( θ ) + ε 2 S (2) f ( θ ) + o ( ε 2 ) , θ [ - π , π ] .

    An explicit formula for the function S(2)f is obtained, based in part on the bounds in (i).

The results are illustrated using a version of the timing channel of Anantharam and Verdu.