Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is there any link between the law of maximum entropy and the principle of least action. Is it possible to derive one from the other ?

share|cite|improve this question
This is annoying! It seems to be a very good question and we have two conflicting answers with same points; and yet, the question seems to have lost interest :( Someone please start a bounty, please. – mehfoos Jul 23 '13 at 16:33
some related links, here and Jaynes's maxent and action, also check the generalised formulations of entropy (2dd law) for both equilibrium and non-quilibrium thermodynamics – Nikos M. Jun 17 '14 at 17:21
Related: – Qmechanic Nov 15 '15 at 14:40

Yes, there is a link, both are examples of extremum principles. And yes it is possible to derive the principle of least action from the law of maximum entropy. The derivation is lengthy and I will only sketch the main steps needed:

  1. We start from the law of maximum entropy $dS/dt \geq 0$. As we know this law is only valid for isolated systems [i]. For dissipative systems $dS/dt > 0$, the evolution is irreversible and cannot be described by an action principle. We must consider non-dissipative systems, for which $dS/dt = 0$. This is correct because the action principles are rigorously restricted [ii] to Nondissipative Systems.

  2. From the phase space structure we can show that the phase space state $\rho$ satisfies the equation $d\rho/dt = \partial\rho/\partial t - \mathcal{L}\rho$, where $\mathcal{L}$ is the Liouvillian.

  3. From the constancy of entropy (1), we can derive the Liouville theorem $d\rho/dt=0$, using the Gibbs relation $S=S(\rho)$. This implies that the general equation of motion (2) reduces to the Liouville equation $\partial\rho/\partial t = \mathcal{L}\rho$. Effectively, this equation is not dissipative and conserves entropy.

  4. For a mechanical system in a pure state, the phase space state is given by the well-known product of Dirac deltas; substituting this $\rho_\mathrm{pure}$ on the Liouville equation, the equation reduces to the Hamilton equations of motion: $dq/dt = \partial H / \partial p$ and $dp/dt = -\partial H / \partial q$.

  5. Using the Hamilton Jacobi method, the Hamilton equations of motion can be written again as a single equation: the Hamilton Jacobi equation $H + \partial A / \partial t = 0$, where $A$ is the action [iii].

  6. It can be shown that the Hamilton Jacobi equation "is an equivalent expression of an integral minimization problem such as Hamilton's principle", and Hamilton's principle is just the Hamiltonian version of the principle of least action. In other words, solving the Hamilton Jacobi equation one obtains the action $A$ and this automatically satisfies the principle of least action $\delta A=0$.

  7. Other versions of the principle of least action can be obtained from here. For instance, the Lagrangian version of the principle can be obtained using a Legendre transformation for deriving the Lagrangian $L = pv - H$. In this case, the action is given by $A=\int L dt$.

[i] For non-isolated system entropy can increase, decrease or remain constant.

[ii] As reported in the Scholarpedia link there are some few special dissipative systems which can be described by an action principle. Those are open systems for which the production of entropy is compensated by an external flow of entropy to give a zero total variation $dS=d_iS+d_eS=0$. Moreover, the action principle only describes the average behaviour of these systems, but not the thermal fluctuations.

[iii] References on mechanics usually denote the action by $S$, but here it would be confused with entropy.

share|cite|improve this answer
I don't know how you derive Liouville's Theorem from the constancy of the entropy. Further, you have already limited the discussion to an isolated system and the entropy is more general than that. – Paul J. Gans Dec 26 '12 at 19:51
@juanrga: Both your accounts are unregistered. You may need to add a login (click "my logins" on your profile) to make it a registered account. Unregistered accounts expire with the browser session. If you want your accounts to be merged, add "merge with <X>" on both profile pages – Manishearth Dec 26 '12 at 20:36
@PaulJ.Gans $dS/dt = -k\int dp dq (\ln \rho + 1) d\rho/dt$. Yes I limited the discussion to isolated systems, because the OP asked for derivation from "the law of maximum entropy" – juanrga Dec 27 '12 at 19:09
Technically everything seems to be correct and indeed answers half of the OP's question. – Yrogirg Dec 30 '12 at 17:52
Concerning the second point --- how to you derive the form of $\mathcal L$? Shouldn't you here effectively postulate Hamiltonian mechanics? – Yrogirg Dec 31 '12 at 3:51

"Entropy" and "action" are two entirely different concepts. The first relates to a coarse-grained statistical description of a physical system at macroscopic scales, the latter to the underlying deterministic microscopic dynamics exhibited by the system.

Also note that: 1) the second law of thermodynamics tells us entropy doesn't decrease, it need not increase and certainly can attain non-maximal values, and 2) action is stationary and not necessarily minimal or maximal. Hence, when considered as fundamental physics laws, both 'maximal entropy' and the 'principle of least action' are misnomers.

Zooming in to the core of your question: 'entropy increase' and 'action stationarity' are unrelated, and even incompatible. One certainly can not be derived from the other. This is for the simple reason that 'action stationarity' describes a reversible physics, while 'entropy increase' presents us with an irreversible picture of evolution of physical systems. The difference, again, is in microscopic versus macroscopic.

As an analogy, think about two statements one can make about the physics of pool billiard. The first being that the balls collide according to Newtons laws which can be expressed by stating that the detailed balls trajectories render a quantity called 'action' stationary. The second being the coarse-grained statistical statement that as long as balls aren't pocketed yet, the mean distance between the balls doesn't decrease. Both statements are unrelated and apply to a description of pool billiard at different levels.

share|cite|improve this answer
There are many issues in this answer. For instance, it confounds the general concept of "entropy" with the approx. macroscopic entropy used in classical thermodynamics. The entropies used in nanothermodynamics, thermodynamics of small systems and in quantum thermodynamics are not defined "at macroscopic scales"... – juanrga Dec 30 '12 at 14:15
Not sure what exact problems you have with my answer. Reading it again, I have used the oportunity to expand it significantly. Let me know in case you believe anything in my answer is incorrect or didactically sub-optimal. – Johannes Dec 30 '12 at 16:51
As said "there are many" I gave one example in my previous post, which you have ignored. Your edit adds more misunderstandings and false statements. E.g., your recent "the second law of thermodynamics tells us entropy doesn't decrease" is a typical misunderstanding of the second law repeated by anti-evolucionists for instance... – juanrga Dec 31 '12 at 11:59

This old russian paper seemed to have discussed the problem without going much into non-equilibrium statistical mechanics. I haven't read the entire thing. Soviet Physics Journal May 1991, Volume 34, Issue 5, pp 426-431. Moslov is the author. Check out their other papers too.

share|cite|improve this answer

As Joannes says, the two principles belong to two different theories:

  • the least action principle is a principle about the (conservative) laws of motion and a proposition about the paths actually followed by the degrees of freedom of mechanical bodies

  • the maximum entropy principle refers either to thermodynamics to figure out in which direction will a transformation occur or more generally to Bayesian inference theory and is rarely concerned with dynamics

They are thus quite different beasts conceptually and, although there may be some overlaps between the two in some instances, I would not recommend, as a matter of principle, to think that these two things are closely related as it is not the case.

As for "deriving" one from the other, at the very least we should agree first about what are the axioms we start from.

For instance, some versions of Crook's fluctuation theorem enable one to get something very close to a maximum entropy principle (on average) "from" simply hamiltonian mechanics (which I would put on the same footing as the least action principle for now) but it is clear that the theorem relies a lot on probability theory (and possibly on Bayesian inference) which, in my view at least, is outside the scope of the least action principle alone.

I think that's an important shift to notice even when invoking Liouville's theorem which is a theorem that makes propositions about probability densities and not about trajectories.

share|cite|improve this answer

I am sorry for the joke about consciousness. Jokes sometimes lower entropy; apparently did not succeed this time. There is indeed a lot of confusion related to the second law. The "proof" of Jaynes as many others have additional assumptions that seem innocuous. They are not. The best is to depart from too general theorems, since in difference to math. in physics they very often come with external assumptions that seem as I said innocuous, but difficult to assess rigorously.Let us consider a particular case of Hamiltonian equations; the Euler equations with Euler fictitious force. There are no rigorously ideal fluids in nature, but for many problems fluid mechanical problems can be treated with the EQ with zero viscosity. But this is a rigorously Hamiltonian equation with uncountable number of topological invariants and as many steady state solutions. The real equations to describe fluid flows is empirical Navier-Stokes equations, the NSE with viscous term in the r.h.s. In a steady state and assuming the flow in the infinite domain and the fluid having nonzero but tending to zero viscosity the flow will be highly turbulent, because the NSE are scale invariant and the flows nature is determined by a dimensionless parameter proportional to inverse viscosity. Since it tends to zero this parameter tends to infinity and this means turbulent flow; in simlest case homogeneous and isotropic. The flow is stirred at some large scale coherently, say by the coherent paddles of the same size. Therefore the initial forced flow has just one harmonic, it is perfectly coherent. The fictitious Euler force however splits this coherent motion into large number of vortices of sequentially smaller and smaller scales. Eventually they will become so small that the viscous term in the r.h.s. becomes large even though the viscosity is arbitrary small. the number of the small vortices will tend to infinity as inverse part of the viscosity power 3/4 if I am not mistaken. Obviously the number of degrees of freedom hat was one for the coherent large scale motion is now infinite. And they are mainly uncorrelated; this is not really true but they almost uncorrelated. In the kinetic sense their entropy as just the number of degrees of freedom is huge and all this now chaotic energy dissipates into heat, a totally chaotic infrared heat. So what happened? I will explain what happened tomorrow. In the meantime I just mention that there is no analytic transition from viscosity tending to zero and viscosity equal to zero. If not for this nonanalyticity fish would not swim and birds and aircrafts would not fly; remarkably the lift force does not depend on viscosity; but this is an illusion because if viscosity was zero there would be no lift at all. In fact the ideal fluid flows are just the uncountable e number of diffeomorphisms and since there can be no observer immersed into ideal fluid

share|cite|improve this answer
This is very hard to follow. Can you work on spacing the ideas out into paragraphs? – Brandon Enright Jul 1 '15 at 21:19

Its a long time controversy. The Hamiltonian time reversible equations are as a matter of common sense not capable and incompatible with the second law. Moreover if in any system like Earth or Cosmos the growth of order is observed it means that the system is open and fed with negative entropy flux by a source of coherent momentum/energy. Moreover if the system does not increase the global energy it means that the coherent energy and negative entropy flux is compensated by dissipation of chaotic energy with many more uncorrelated degrees of freedom, while the global "order" of the system grows, or its global entropy diminishes. This is a kinetic process when a system is open with external to this system coherent source of momentum/energy and having a sink for disposal of chaotic, weakly correlated degrees of freedom. Such kinetic systems and processes are not subject to time reversible Hamiltonian equations of physics. Its obvious; basta. Unfortunately all systems in Cosmos and Cosmos itself are open systems. Earth is obviously one (read Schroudinger 1944, What is Life. But Cosmos is also an open system. The acceleration of Hubble expansion is an incontrovertible proof. Its hard to understand how some don't see it. This is not just the positive sign negative pressure fantasmagory of cosmological constant. It is dark energy source of coherent momentum/energy flux into our Cosmos which is a subdomain embedded into the timeless continuum of dark energy. The global energy is dissipated as chaotic at the Planck scale; the MBH is the sink. Order stays in our Cosmos and is growing from the BB. This is why baryon matter and we exist and are concerned with the kinetic second law and some even with the meaning of life. Look at (E. Levich, 2014 and 2015) for the calculations and elaborations; also for enhancement of consciousness.

Eugene Levich

share|cite|improve this answer
This does not seem to clearly make a physics-based argument around the question. It also seems to be heading directly towards non-mainstream physics when mentioning, e.g., 'the enhancement of consciousness'. If you wish to edit and focus on the physics of the question, please do so. – Jon Custer Jun 30 '15 at 13:33
Formatting breaks might also help; the markdown on Stack Exchange uses two lines for paragraph breaks, rather than the normal one line. – Kyle Kanos Jun 30 '15 at 14:17
"The Hamiltonian time reversible equations are as a matter of common sense not capable and incompatible with the second law" This is a common misconception, maintained by misunderstanding of the second law and misidentifying $\int -\rho\log \rho\, dqdp$ with quantity that is always supposed to describe thermodynamic entropy. Jaynes has shown how to derive 2nd law from Hamiltonian mechanics and other additional assumptions (probabilistic and experimental, which do not belong to mechanics). – Ján Lalinský Jun 30 '15 at 20:53
I am sorry for the joke about consciousness. Jokes sometimes lower entropy; apparently did not succeed this time. – eugene levich Jul 1 '15 at 19:46

protected by Qmechanic Jul 1 '15 at 21:14

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.