Stationary Distribution

A probability distribution $\pi$ over the state space $S$ (written as a row vector) is a stationary distribution for a Markov chain with transition matrix $P$ if $\pi P=\pi$, equivalently, ${\pi }_j={\sum }_i{\pi }_i{p}_{ij}$ for every state $j$, together with ${\sum }_j{\pi }_j=1$. For an irreducible, positive recurrent chain, a stationary distribution exists and is unique, and ${\pi }_i=1/{\mathbb{E}}_i[{\tau }_i]$, the reciprocal of the expected return time to state $i$. If the chain is also aperiodic (i.e. ergodic), the state distribution converges to $\pi$ from any starting distribution as $n\to \infty$, making $\pi$ both the equilibrium and the long-run limiting distribution. Not to be confused with 'stationarity' in the time-series sense (an unchanging joint distribution of a sequence over time), the Markov-chain stationary distribution is a distinct, though related, concept describing equilibrium over the state space.

• The equation $\pi P=\pi$ says $\pi$ is a left eigenvector of $P$ with eigenvalue 1, computing the stationary distribution is, in the finite-state case, literally an eigenvector problem, which is why linear algebra tools apply directly to Markov chain analysis.
• A stationary distribution can exist for a chain that is not ergodic (e.g. a periodic chain still has one), what ergodicity adds on top is the guarantee that the chain actually converges to $\pi$ from any starting point, rather than merely having $\pi$ as a fixed point.
• MCMC algorithms are built by deliberately reverse-engineering a Markov chain whose stationary distribution equals a target distribution of interest, such as a Bayesian posterior, detailed balance is the standard tool used to guarantee the constructed chain has exactly the desired stationary distribution.