Section: New Results
Ergodic theory for controlled Markov chains with stationary inputs
Consider a stochastic process on a finite state space . It is conditionally Markov, given a real-valued `input process' . This is assumed to be small, which is modeled through the scaling, where is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on :
A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain obtained with . The triple is a jointly stationary process satisfying Moreover, a second-order Taylor-series approximation is obtained:
The results are illustrated using a version of the timing channel of Anantharam and Verdu.