# Markov Process in Monte Carlo simulation

I am trying to understand the detailed balance condition precisely.

It is written the following :

Let $P(A,t)$ be the probability distribution to be in state $A$ at time $t$.

We have :

$$P(A,t+1)=P(A,t)+\sum_B W(B \rightarrow A)P(B,t) - W(A \rightarrow B)P(A,t)$$

Where $W(B \rightarrow A)$ is the probability that the system goes from state $B$ to state $A$.

I don't understand this formula.

I would write this :

$$P(A,t+1)=\sum_B P(B,t) W(B \rightarrow A,t)$$

Indeed, consider I am at time $t+1$. The only things I need to know the probability of where I can be at time $t$ are given by the previous states + the probability of transition.

I don't get why we would substract things like the formula above...?

Let's say that in time $t+1$ you are in state $A$. You don't know where you have been before, but you have two options:

• You were in state $B$ at $t$ and jumped into $A$. Probability you were in $B$ is $P(B,t)$. The jump happens with probability $W(B\rightarrow A)$, so the total one is $P(B,t)W(B\rightarrow A)$.

• You were already in $A$ and nothing happened. The probability of being in $A$ at $t$ is $P(A,t)$. And the probability that nothing happened during the time interval (i.e. you did not jumped out of the state $A$ during the time interval) is $(1-W(A\rightarrow B))$. Then, the probability for this option is $P(A,t)\cdot(1-W(A\rightarrow B))$.

So, you have option one or option two:

$$P(A,t+1)=P(B,t)W(B\rightarrow A) + P(A,t)(1-W(A\rightarrow B))$$.

Add a sum over all possible $B$ states, make the product of the last term, and you recover your formula.

• Thank you for your answer. In fact from my perspective the $W(B \rightarrow A,t)$ already contain the case when nothing change or when the state "when out" from $A$. In a sense, I have my $P(A,t)*W(A \rightarrow A)=P(A,t)*(1-\sum_B W(A \rightarrow B))$ where lhs is my point of view and rhs yours. So I guess that the summation from the link suppose that $B \neq A$ right (whereas in my case $B=A$ is included) ? Oct 12 '17 at 14:31
• Ok my comment is probably not really clear. I would like to check if indeed the summation on the website assume $B \neq A$ and then they in fact say the exact same thing than me. That was probably what you meant by your last sentence but I am not sure if you talked about my own formula or you said that what you wrote was the same as the website formula. Oct 12 '17 at 14:45
• The $B=A$ case is considered but drops out since there is a $+P(A\rightarrow A) P(A)$ (from first term in sum) and a $-P(A\rightarrow A) P(A)$ (from the second term in sum).
– Dave
Oct 12 '17 at 19:47
• When I have faced these proofs, it is usually assumed that $B\neq A$. Basically each one is a different state, and you can jump between them (as an analogy, think in a quantum particle randomly jumping between energy levels). In addition to that, I would say $W(A\rightarrow A)=0$, since when you go out of $A$, you go somewhere else. The case probability you stayed is 1 minus the probability of going out: $1-W(A\rightarrow B)$. Oct 13 '17 at 8:42