# Markov chain and master equation

I'm trying to understand the meaning (plausibility) of the master equation and of the detailed balance in a Markov stochastic process.

The master equation is formulated as follows (from Landau and Binder, MC Simulations in Statistical Physics, 2009)

$\displaystyle\frac{\mathrm{d} P(S_j, t)}{\mathrm{d} t} = - \sum_i W(S_j\to S_i) P(S_j,t)+\sum_i W(S_i\to S_j)P(S_i,\, t)$

where $P(S_j, t) = P(X_{t_n}=S_j)$ and $W(S_j\to S_i) = P(X_{t_n}=S_i|X_{t_{n-1}}=S_j)$.

However, I still cannot understand its meaning completely.

According to this formulation, one should be able to rewrite it as

$\displaystyle\frac{\mathrm{d} P(S_j, t)}{\mathrm{d} t} = - \sum_i P(S_i,t+\mathrm{d}t)+\sum_i P(S_j,\, t+\mathrm{d}t)$,

right? This, however does not help me more (should the second term on the right be at $t$ instead of $t+\mathrm{d}t$?)

In the book they then give this short explanation

[The master equation] can be considered as a 'continuity equation', expressing the fact that the total probability is conserved ($\sum_j P(S_j,t)\equiv 1$ at all times) and all probability of a state $i$ that is 'lost' by transition to state $j$ is gained in the probability of that state, and vice versa. [It] just describes the balance of gain and loss processes: since the probabilities of the events $S_j \to S_{i_1}$, $S_j \to S_{i_2}$, [...] are mutually exclusive, the total probability for a move away from the state $j$ is simply the sum $\sum_i W(S_i\to S_j)P(S_j,t)$.

[should it not be $\sum_i W(S_j\to S_i)P(S_j,t)$?]

What I do not understand well, is the concept of 'loosing' probability and what does $\frac{\mathrm{d} P(S_j, t)}{\mathrm{d} t}$ 'represent'. Can anyone help me in this regard?

• Your definition of $W$ is incorrect, it is a measure of rate (dimension of per unit time), e.g. $W(S_j\to S_i)=P(X_{t_n}=S_i|X_{t_{n-1}}=S_j)/(t_{n}-t_{n-1})$. Does that make things clearer? Aug 16, 2017 at 12:13

$P(S_j,t)$ is the probability of the system to be in state $S_j$ at time $t$. Therefore, $\frac{dP(S_j,t)}{dt}$ is simply the rate of change of the probability of the system to be in state $S_j$ at time $t$. In other words, knowing $\frac{dP(S_j,t)}{dt}$ tells you whether it is becoming more likely or less likely for the system to be in the given state as time goes on.
The notion of "loosing" probability might be thought of as the probability of being in the state $S_j$ to be decreasing. The idea being that the system must be in some state, so if you sum over the probability of all states then you get unity. That means that whenever one state is losing probability (becoming less likely) others must be gaining probability (becoming more likely).
However, that is not the same as detailed balance. The rate of change of the probability of being in a given state is equal to the probability of transitioning into that state at the given time minus the probability of transitioning out of that state at the given time. That intuitive description is stated mathematically in the so-called master equation. The master equation alone does not guarantee that detailed balance will be satisfied. Detailed balance will only be satisfied when the probability of transitioning into any given state is equal to the probability of transitioning out. Or in other words, when $\frac{dP(S_j,t)}{dt}=0$.