Section: New Results
Ergodic theory for controlled Markov chains with stationary inputs
Consider a stochastic process $\mathbf{X}$ on a finite state space $X=\{1,\cdots ,d\}$. It is conditionally Markov, given a realvalued `input process' $\zeta $. This is assumed to be small, which is modeled through the scaling, ${\zeta}_{t}=\epsilon {\zeta}_{t}^{1},\phantom{\rule{0.277778em}{0ex}}0\le \epsilon \le 1\phantom{\rule{0.166667em}{0ex}},$ where ${\zeta}^{1}$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $\zeta $:

A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain ${\mathbf{X}}^{\u2022}$ obtained with $\zeta \equiv 0$. The triple $(\mathbf{X},{\mathbf{X}}^{\u2022},\zeta )$ is a jointly stationary process satisfying $\U0001d5af\{X\left(t\right)\ne {X}^{\u2022}\left(t\right)\}=O\left(\epsilon \right).$ Moreover, a secondorder Taylorseries approximation is obtained:
$\U0001d5af\{X\left(t\right)=i\}=\U0001d5af\{{X}^{\u2022}\left(t\right)=i\}+{\epsilon}^{2}\varrho \left(i\right)+o\left({\epsilon}^{2}\right),\phantom{\rule{1.em}{0ex}}1\le i\le d,$with an explicit formula for the vector $\varrho \in {\Re}^{d}$.

For any $m\ge 1$ and any function $f:\{1,\cdots ,d\}\times \Re \to {\Re}^{m}$, the stationary stochastic process $Y\left(t\right)=f\left(X\right(t),\zeta (t\left)\right)$ has a power spectral density ${\text{S}}_{f}$ that admits a second order Taylor series expansion: A function ${{\text{S}}^{\text{(2)}}}_{f}:[\pi ,\pi ]\to {C}^{m\times m}$ is constructed such that
${\text{S}}_{f}\left(\theta \right)={\text{S}}_{f}^{\u2022}\left(\theta \right)+{\epsilon}^{2}{{\text{S}}^{\text{(2)}}}_{f}\left(\theta \right)+o\left({\epsilon}^{2}\right),\phantom{\rule{1.em}{0ex}}\theta \in [\pi ,\pi ].$An explicit formula for the function ${{\text{S}}^{\text{(2)}}}_{f}$ is obtained, based in part on the bounds in (i).
The results are illustrated using a version of the timing channel of Anantharam and Verdu.