EN FR
EN FR
New Results
Bilateral Contracts and Grants with Industry
Bibliography
New Results
Bilateral Contracts and Grants with Industry
Bibliography


Section: New Results

Ergodic theory for controlled Markov chains with stationary inputs

Consider a stochastic process 𝐗 on a finite state space X={1,,d}. It is conditionally Markov, given a real-valued `input process' ζ. This is assumed to be small, which is modeled through the scaling, ζt=εζt1,0ε1, where ζ1 is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on ζ:

  • A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain 𝐗 obtained with ζ0. The triple (𝐗,𝐗,ζ) is a jointly stationary process satisfying 𝖯{X(t)X(t)}=O(ε). Moreover, a second-order Taylor-series approximation is obtained:

    𝖯{X(t)=i}=𝖯{X(t)=i}+ε2ϱ(i)+o(ε2),1id,

    with an explicit formula for the vector ϱd.

  • For any m1 and any function f:{1,,d}×m, the stationary stochastic process Y(t)=f(X(t),ζ(t)) has a power spectral density Sf that admits a second order Taylor series expansion: A function S(2)f:[-π,π]Cm×m is constructed such that

    Sf(θ)=Sf(θ)+ε2S(2)f(θ)+o(ε2),θ[-π,π].

    An explicit formula for the function S(2)f is obtained, based in part on the bounds in (i).

The results are illustrated using a version of the timing channel of Anantharam and Verdu.