Ergodic Markov Chain

A Markov chain is ergodic if it is irreducible, aperiodic, and positive recurrent (for a finite state space, irreducibility and aperiodicity alone imply positive recurrence, so the condition simplifies to just those two). An ergodic chain has a unique stationary distribution $\pi$, and the $n$-step transition probabilities converge to it regardless of starting state, $p_{ij}^{(n)}\to {\pi }_j$ as $n\to \infty$ for every $i,j$. Ergodicity also delivers a law-of-large-numbers-style guarantee (the ergodic theorem for Markov chains): the long-run time-average of a function along a single realized chain trajectory converges almost surely to its expectation under $\pi$, $\frac{1}{n}{\sum }_{t=1}^{n}f(X_t)\to {\mathbb{E}}_{\pi }[f(X)]$. This second guarantee is precisely why a single, sufficiently long MCMC chain can be used to estimate posterior expectations.

• Ergodicity delivers two related but distinct guarantees: convergence of the state distribution to $\pi$ regardless of starting point, and a law-of-large-numbers-style guarantee that a single long trajectory's time-average matches the $\pi$-expectation, the second is what makes practical MCMC estimation valid from one chain.
• In practice, MCMC algorithm designers don't verify ergodicity from first principles for every model, standard algorithms like Metropolis-Hastings and Gibbs sampling are constructed with built-in irreducibility and aperiodicity guarantees under mild, generally-satisfied conditions on the target distribution.
• Even a theoretically ergodic chain can converge to its stationary distribution extremely slowly in practice (poor 'mixing'), this is why MCMC diagnostics like trace plots and effective sample size exist, ergodicity guarantees eventual convergence, not fast convergence.