Tell me more ×
Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free, no registration required.
up vote 4 down vote favorite

To my (basic) understanding a Markov process is a process wherein the future state of a system only depends on the current state, and not on the past states of the system.

I was wondering on what the standard approach is on systems that inherently hold information about their past in them.

One could image, in the discrete-time case, system carrying in it a representation of it's three previous states. There are two cases to distinguish here:

  • The future state of the system is not a function of this memory and only of factors that don't have anything to do with it. This is perhaps a boring case (although there might be interesting examples I don't know of).
  • The future state of the system is a function of this memory information, yet in a "Markovian" way. This is the case I am most interested in.

Are these things that would be considered as Markovian systems (perhaps on a different scale)? Are there examples of these kinds of systems?

share|improve this question
2  
In what sense are systems that belong to the second case then "Markovian"...? If their evolution is a function of the memory of the system, then they cannot be Markovian by definition. However, if this amount of memory is finite (like the example of storing 3 previous states that you mention), then on time-scales long compared to this memory-storage time scale, the system would be effectively Markovian. – Vijay Murthy Mar 11 '12 at 19:33
1  
A definite example of what I said above is the generalized Langevin equation. See here and here for a discussion and also the Markovian approximation. – Vijay Murthy Mar 11 '12 at 19:46
You can do the product of the system state space and the memory state space, getting a Markov process. But maybe you are thinking about something different, with additional restrictions on the allowed transitions. – mmc Mar 11 '12 at 20:54
I guess what I'm asking is this: are the restrictions for an evolving system to be Markovian laid mainly on the system or on the state-to-state transition rules? And why the one or the other? Perhaps this is too axiomatic to be questioned even, in which case that would be an answer too. – romeovs Mar 11 '12 at 21:09

1 Answer

up vote 5 down vote accepted

Yes. The same system can - at least in many cases - be described by either a stochastic process with memory or by a Markov process. The point is that in order to write it as a Markov process, one must add enough variables encoding the memory. For example, an autoregressive moving average (ARMA) process is defined as a process with memory, but each such process has an equivalent representation as a linear state space model, which is Markovian.

Thus, from a practical point of view, it is a matter of convenience, or of what you are trying to achieve.

If you want a minimal representation with the fewest number of variables, you usually need memory, while if you want a Marovian model (often the more flexible and computationally more attractive choice), you need a big model.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.