Recurrent State

A state $i$ is recurrent if, starting from $i$, the probability of eventually returning to $i$ is 1: $P(\text{return to }i\mid X_0=i)=1$. By the strong Markov property, a recurrent state is visited infinitely often with probability 1. Recurrence is further classified as positive recurrent, if the expected return time is finite, or null recurrent, if the return probability is 1 but the expected return time is infinite (this distinction only arises for infinite state spaces). Positive recurrence of every state in an irreducible chain is precisely the condition needed to guarantee existence of a unique stationary distribution.

• In a finite-state, irreducible Markov chain every state is automatically recurrent (in fact positive recurrent), the null-recurrent case only arises in infinite state spaces, such as a simple symmetric random walk on the integers.
• Recurrence is a class property within an irreducible chain: if one state is recurrent, every state communicating with it is also recurrent, so for irreducible chains it suffices to check just one state.
• The distinction between positive and null recurrence has a famous illustration in random walk theory: the simple symmetric random walk on $\mathbb{Z}$ (one dimension) and ${\mathbb{Z}}^2$ (two dimensions) is null recurrent, it returns to the origin with probability 1 but takes, on average, infinitely long to do so, while the same walk in three or more dimensions is transient altogether (Pólya's recurrence theorem).