EN FR
EN FR

## Section: New Results

### Ergodic theory for controlled Markov chains with stationary inputs

Consider a stochastic process $𝐗$ on a finite state space $X=\left\{1,\cdots ,d\right\}$. It is conditionally Markov, given a real-valued `input process' $\zeta$. This is assumed to be small, which is modeled through the scaling, ${\zeta }_{t}=\epsilon {\zeta }_{t}^{1},\phantom{\rule{0.277778em}{0ex}}0\le \epsilon \le 1\phantom{\rule{0.166667em}{0ex}},$ where ${\zeta }^{1}$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $\zeta$:

• A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain ${𝐗}^{•}$ obtained with $\zeta \equiv 0$. The triple $\left(𝐗,{𝐗}^{•},\zeta \right)$ is a jointly stationary process satisfying $𝖯\left\{X\left(t\right)\ne {X}^{•}\left(t\right)\right\}=O\left(\epsilon \right).$ Moreover, a second-order Taylor-series approximation is obtained:

$𝖯\left\{X\left(t\right)=i\right\}=𝖯\left\{{X}^{•}\left(t\right)=i\right\}+{\epsilon }^{2}\varrho \left(i\right)+o\left({\epsilon }^{2}\right),\phantom{\rule{1.em}{0ex}}1\le i\le d,$

with an explicit formula for the vector $\varrho \in {\Re }^{d}$.

• For any $m\ge 1$ and any function $f:\left\{1,\cdots ,d\right\}×\Re \to {\Re }^{m}$, the stationary stochastic process $Y\left(t\right)=f\left(X\left(t\right),\zeta \left(t\right)\right)$ has a power spectral density ${\text{S}}_{f}$ that admits a second order Taylor series expansion: A function ${{\text{S}}^{\text{(2)}}}_{f}:\left[-\pi ,\pi \right]\to {C}^{m×m}$ is constructed such that

${\text{S}}_{f}\left(\theta \right)={\text{S}}_{f}^{•}\left(\theta \right)+{\epsilon }^{2}{{\text{S}}^{\text{(2)}}}_{f}\left(\theta \right)+o\left({\epsilon }^{2}\right),\phantom{\rule{1.em}{0ex}}\theta \in \left[-\pi ,\pi \right].$

An explicit formula for the function ${{\text{S}}^{\text{(2)}}}_{f}$ is obtained, based in part on the bounds in (i).

The results are illustrated using a version of the timing channel of Anantharam and Verdu.