Semi-Markov Process

A semi-Markov process is defined by an embedded Markov chain $\{Y_n\}$ describing the sequence of states visited, together with holding times $\{T_n\}$ where $T_n$, the time spent in state $Y_n$ before jumping to $Y_{n+1}$, may follow an arbitrary probability distribution $F_{ij}(t)$ that can depend on both the current state $i=Y_n$ and the next state $j=Y_{n+1}$. Formally, the process is defined by the semi-Markov kernel $Q_{ij}(t)=P(Y_{n+1}=j,T_n\le t\mid Y_n=i)$. The process $X_t$ at continuous time $t$ is simply whichever state $Y_n$ is currently active. When every holding-time distribution is exponential, a semi-Markov process reduces exactly to a continuous-time Markov chain; when every holding time is fixed (deterministic) at 1, it reduces to a discrete-time Markov chain evaluated at each unit of time. The full process $X_t$ is generally not itself Markov in continuous time (because how long the process has already been in the current state can affect how much longer it stays), which is precisely the modeling flexibility being traded for lost memorylessness.

• The process $X_t$ itself, unlike its embedded jump chain, is generally not Markov: knowing how long the process has already spent in its current state can be informative about how much longer it will stay, since the holding-time distribution need not be memoryless.
• Reliability engineering and maintenance modeling are classic applications: a machine's time-to-failure or time-to-repair is rarely well modeled by the exponential distribution's constant-hazard assumption, semi-Markov models let engineers plug in more realistic distributions like Weibull or gamma.
• Because full generality (arbitrary holding-time distributions) sacrifices the clean matrix-exponential machinery available for CTMCs, semi-Markov models are typically analyzed via renewal-theory techniques or simulation rather than closed-form solutions.