How does time in a discrete Markov Chain relate to physical time? Given some arbitrary discrete-time Markov Chain, is there a way to relate the time of the model to physical values in units of seconds?
For instance, I've constructed a model for diffusing particles in a box of linear size $L$. The particles have diffusion constant $D$, and there is a natural timescale arising from dimension analysis $L^2/D$. Is there a way to relate such a physical timescale to the discrete timesteps of a Markov Chain?
To be a bit more explicit, the system I consider is a box of $N$ indistinguishable particles which is divided into $R$ different compartments. The states of the Markov chain are defined by the combinations of dividing these $N$ particles into the $R$ compartments. However, I want the transition rates to depend on physical parameters of the system, including a timescale which I need to relate to the time in a Markov Chain.
 A: $L^2/D$ is roughly the time required for your particles to have diffused over a distance $L$. You must use it as a time-scale and work in terms of non-dimensional time steps for your Markov chain: $\Delta t^*=\Delta t D/L^2$. You will want to make your time steps small, but "small" is meaningful only when you speak of non-dimensional quantities such as $\Delta t^*$. By taking a small time step in your Markov chain, you may be able to introduce some physically realistic hypotheses about how present state affects future state (one time step ahead) in your Markov chain, and thus arrive at a physically sensible transition matrix. These hypotheses that you shall introduce, will ultimately determine things such as transition rate in terms of your diffusion time scale and any other physical parameters appearing in the hypotheses.
Markov chains etc. are mathematical objects and they by themselves will not tell you anything about the physical world. Your mettle as a scientist now consists in what physical hypotheses you shall introduce to determine the transition matrix.
