127
$\begingroup$

I've just finished a Classical Mechanics course, and looking back on it some things are not quite clear. In the first half we covered the Lagrangian formalism, which I thought was pretty cool. I specially appreciated the freedom you have when choosing coordinates, and the fact that you can basically ignore constraint forces. Of course, most simple situations you can solve using good old $F=ma$, but for more complicated stuff the whole formalism comes in pretty handy.

Then in the second half we switched to Hamiltonian mechanics, and that's where I began to lose sight of why we were doing things the way we were. I don't have any problem understanding the Hamiltonian, or Hamilton's equations, or the Hamilton-Jacobi equation, or what have you. My issue is that I don't understand why would someone bother developing all this to do the same things you did before but in a different way. In fact, in most cases you need to start with a Lagrangian and get the momenta from $p = \frac{\partial L}{\partial \dot{q}}$, and the Hamiltonian from $H = \sum \dot{q_i}p_i - L$. But if you already have the Lagrangian, why not just solve the Euler-Lagrange equations?

I guess maybe there are interesting uses of the Hamiltion formalism and we just didn't do a whole lot of examples (it was the harmonic oscillator the whole way, pretty much). I've also heard that it allows a somewhat smooth transition into quantum mechanics. We did work out a way to get Schrödinger's equation doing stuff with the action. But still something's not clicking.

My questions are the following: Why do people use the Hamiltonian formalism? Is it better for theoretical work? Are there problems that are more easily solved using Hamilton's mechanics instead of Lagrange's? What are some examples of that?

$\endgroup$

15 Answers 15

83
$\begingroup$

There are several reasons for using the Hamiltonian formalism:

1) Statistical physics. The standard thermal states weight pure states according to

$$\text{Prob}(\text{state}) \propto e^{-H(\text{state})/k_BT}$$

So you need to understand Hamiltonians to do stat mech in real generality.

2) Geometrical prettiness. Hamilton's equations say that flowing in time is equivalent to flowing along a vector field on phase space. This gives a nice geometrical picture for how time evolution works in such systems. People use this framework a lot in dynamical systems, where they study questions like 'is the time evolution chaotic?'.

3) Generalization to quantum physics. The basic formalism of quantum mechanics (states and observables) is an obvious generalization of the Hamiltonian formalism. It's less obvious how it's connected to the Lagrangian formalism, and way less obvious how it's connected to the Newtonian formalism.

[Edit to answer Javier's comment] This might be too brief, but the basic story goes as follows:

In Hamiltonian mechanics, observables are elements of a commutative algebra which carries a Poisson bracket $\{,\}$. The algebra of observables has a distinguished element, the Hamiltonian, which defines the time evolution via $d\mathcal{O}/dt = \{\mathcal{O},H\}$. Thermal states are simply linear functions on this algebra. (The observables are realized as functions on the phase space, and the bracket comes from the symplectic structure there. But the algebra of observables is what really matters: You can recover the phase space from the algebra of functions.)

On the other hand, in quantum physics, we have an algebra of observables which is not commutative. But it still has a bracket $\{,\} = -\frac{i}{\hbar}[,]$ (the commutator), and it still gets its time evolution from a distinguished element $H$, via $d\mathcal{O}/dt = \{\mathcal{O},H\}$. Likewise, thermal states are still linear functionals on the algebra.

$\endgroup$
44
$\begingroup$

Some more comments to add to user1504's response:

  1. For a system with configuration space of dimension $n$, Hamilton's equations are a set of $2n$, coupled, first-order ODEs while the Euler-Lagrange equations are a set of $n$, second-order ODEs. In a given problem it might be easier to solve the first order Hamilton's equations (although sadly, I can't think of a good example at the moment).

  2. It's true that quantum mechanics is usually presented in the Hamiltonian formalism, but as is implicit in user1504's answer, it is possible to use a Lagrangian to quantize classical systems. The Hamiltonian approach is commonly referred to as "canonical quantization", while the Lagrangian approach is referred to as "path integral quantization".

Edit. As user Qmechanic points out, my point 2 is not strictly correct; path integral quantization can also be performed with the Hamiltonian. See, for example, this physics.SE post:

In Path Integrals, lagrangian or hamiltonian are fundamental?

$\endgroup$
  • $\begingroup$ Should that second $2n$ be just $n$? I would hope the Lagrangian method doesn't require twice as many IC's. $\endgroup$ – user10851 Dec 5 '13 at 7:04
  • 2
    $\begingroup$ Comment to the last sentence of the answer (v2): It should be stressed that there also exists a Hamiltonian formulation of the path integral, cf. e.g. this Phys.SE post. $\endgroup$ – Qmechanic Dec 5 '13 at 7:41
  • 4
    $\begingroup$ I have an example for you. First order ODEs are much easier to solve numerically: you can write a vector $x$ with 2n generalized coordinates and momenta, and write $\dot{x} = F(x)$. Then at each point $x$ in phase space, you know which direction to "step" in. You can even analyze stability by linearizing $F$, and looking at the sign of the real parts of the eigenvalues of the jacobian. $\endgroup$ – ZachMcDargh Dec 7 '13 at 17:20
  • 2
    $\begingroup$ A remark to (1): It is always trivial to translate a system of second order equations into first oder equations: Just replace $\ddot x$ by $\dot y$ (introducing the function $y$) and add the equation $y = \dot x$. You don't need Hamilton's formulation to achieve that. Hamilton's equations are however much more symmetrical than the ones I just proposed. $\endgroup$ – Friedrich May 6 '14 at 16:56
  • $\begingroup$ Slight nitpick: the $n$ different second-order ODE's that constitute the Euler-Lagrange equations are in general coupled. As @ZachMcDargh points out, first-order ODE's are much easier to solve numerically. (In fact, when numerically solving a second-order ODE, one almost always immediately converts it into a system of coupled first-order ODE's using the procedure that Friedrich explains.) So either way, in practice you have to solve $2n$ first-order ODEs. But as Friedrich says, Hamilton's equations are more symmetric. $\endgroup$ – tparker Jun 16 '17 at 22:43
23
$\begingroup$
  1. First of all, Lagrangian is a mathematical quantity which has no physical meaning but Hamiltonian is physical (for example, it is total energy of the system, in some case) and all quantities in Hamiltonian mechanics has physical meanings which makes easier to have physical intuition.

  2. In Hamiltonian mechanics you have canonical transformations which allows you change coordinates and find an easier canonical coordinates and momenta in which it is easier to solve problem.

  3. The best of all is, Lagrangian is a powerful mathematical method to solve problems in classical mechanic but Hamiltonian is a powerful method to solve problems in classical mechanics, quantum mechanics, statistical mechanics, thermodynamics... etc actually almost all physics...

For example: In thermodynamics: Gibbs free energy, Helmholtz free energy... are all canonical transformations of Hamiltonian..

$\endgroup$
  • 1
    $\begingroup$ Hamiltonian is not always total energy, though it is in many of the systems one typically studies in a classical mechanics course. $\endgroup$ – BMS Dec 5 '13 at 4:03
  • 1
    $\begingroup$ Yes, it isn't always total energy but my point is "it is always physical." $\endgroup$ – maynak Dec 5 '13 at 4:23
  • 3
    $\begingroup$ Not all Hamiltonian quantities have direct physical meaning. Generalized momenta and coordinates can be mixed functions of Cartesian coordinates and their corresponding momenta, with no guarantee that the result has simple physical interpretation. Also in thermodynamics, Gibbs and Helmholtz functions are not Legendre transformations of Hamiltonian, but of internal energy. For homogeneous system, this is function $U(S,V,N)$, which is related to Hamiltonian from mechanics, but still is a very different thing. $\endgroup$ – Ján Lalinský Dec 5 '13 at 14:23
  • 6
    $\begingroup$ Lagrangian has a clear physcial meaning. That is the difference between the kinetic and potential energy. $\endgroup$ – Dimiter P Jan 8 '16 at 0:25
  • $\begingroup$ Lagrangian has no physical meaning? How come? $\endgroup$ – L.K. Jun 1 '17 at 11:44
15
$\begingroup$

An additional point which was not emphasized by the previous answers enough is that Hamiltonian formalism allows you to do canonical transformations to switch to the best possible coordinate system in phase space to describe the system. This is a lot better than in Lagrangian mechanics, where you can only do coordinate transformations in configuration space. (The phase space has twice the number of dimensions, so you have a larger freedom.) I find Poisson brackets very useful in Hamiltonian mechanics to write the equations of motion of an arbitrary function of phase space variables: $\dot{Q} = \{Q, H\}$. It is possible to find conserved quantities ($\dot{Q}=0$) in Hamiltonian mechanics which are not obvious in Lagrangian mechanics.

Examples:

  1. Normal mode oscillations. If the Hamiltonian turns out to be a quadratic function of coordinates and momenta for a system of $N$ objects, e.g. $H=\sum_{ij} M_{ij} q_i q_j + \sum_{ij} M_{ij} p_i p_j$ then you can simply do a canonical transformation along the eigenvectors of $M_{ij}$ to diagonalize $M_{ij}$, and your system separates into independent harmonic oscillators.

  2. Perturbation theory. You can simply examine the oscillations around the equilibrium state by expanding the Hamiltonian to second order in the phase space variables.

  3. In planetary dynamics, there is a large separation of scales between the interaction of planets with the central star and their mutual interactions. "Secular theory" describes the very long term evolution of the system using Hamiltonian mechanics. You can apply a canonical transformation (Von Zeipel transformation) along the action-angle variables of the short term interactions. You can then derive the long term evolution, (for example that of the eccentricities and inclinations), investigate whether the long-term perturbing effects of planets add up resonantly or not, whether the system is chaotic, etc.

$\endgroup$
  • $\begingroup$ Fantastic. This goes so nicely together with the information theoretic considerations (which led me to this page, ultimately, while thinking about ML and independence). $\endgroup$ – jhegedus May 16 at 6:48
7
$\begingroup$

One way to see the relationship of Hamilotian classical mechanics and Quantum mechanics is not to look for a direct translation of Hamiltionian -> Quantum Hamiltionian (which exists: Geometric Quantization), but consider the reverse relationship. Given a Hamiltion operator and evaluating it on wave functions of the form $e^{\frac{i}{\hbar} \phi}$ (which can be thought as a highly located wave packet) simplifies in the limit $\hbar \to 0$ to the Hamiltion-Jacobi-equation with the classical Hamiltionian. This is known as WKB-approximation and also applies to optics (i.e light rays follow the integral curves of the associated Hamiltionian picture in first approximation).

$\endgroup$
  • $\begingroup$ Could you please specify what is phi here? $\endgroup$ – chris Dec 26 '18 at 21:31
  • $\begingroup$ In this context, it is an arbitrary real-valued function describing the phase of the wave package. The WKB approximation usually yields some kind of equation for it (for the Schrödinger equation it is the Hamilton-Jacobi equation). $\endgroup$ – Tobias Diez Dec 26 '18 at 23:17
7
$\begingroup$

In addition to the several great answers already posted:

1) Hamiltonian mechanics lends itself to a general and systematic form of perturbation theory called "canonical perturbation theory." Perturbation theory in Lagrangian mechanics tends to be a bit more ad hoc and case-by-case. I suspect that this is why Hamilton and Jacobi originally developed the theory, as of course they didn't know about its future stat mech and quantum applications.

2) Hamiltonian mechanics leads to the Hamilton-Jacobi equation, which is useful for finding non-obvious conserved quantities for complicated systems.

3) The Hamilton-Jacobi equation in turn leads to action-angle variables, which are especially useful in astronomy (which the early physicists cared about a lot).

$\endgroup$
  • $\begingroup$ Any exemples of these non obvious quantities in 2)? $\endgroup$ – lalala May 10 '18 at 5:25
  • 1
    $\begingroup$ @lalala: I can answer that now. The Hamilton-Jacobi equation leads to a "hidden" constant of motion for particles moving around a Kerr black hole, which in turn lets you reduce all the equations to first order and do things like calculate the shape of the shadow. $\endgroup$ – Javier May 5 at 13:46
7
$\begingroup$

This is a fact about the hamiltonian compared to the lagrangian which I find not trivial (and worth to keep in mind).

Suppose that the lagrangian $L$ and hamiltonian $H$ are cyclic with respect to some coordinate $q_1$. Then we have a theorem (cfr. [1]):

The evolution of the other coordinates $q_2,...,q_n$ is the one of a system with $n-1$ independent coordinate $q_2,...,q_n$ with Hamiltonian $$H(p_2,...,p_n,q_2,...,q_n,t,c),$$ dependent from the parameter $c=p_1$.

Note that this is false if instead of $H$ we state the theorem for the lagrangian $L$.

To see exactly what I mean, consider the simplified Lagrangian of the two body problem: $$L=\frac{\mu}{2} (\dot r ^2+r^2\dot \varphi ^2 )-U(r).$$ We have $$p_\varphi=\mu r^2 \dot \varphi=\ell \quad(\text {constant}).$$ Now try to plug $$\dot \varphi=\frac{\ell}{\mu r^2}$$ inside the lagrangian and compare the equations of motion so obtained with the ones that you get plugging it directly into the equations of motion $\frac{\partial L}{\partial r}=\frac{d}{dt}\frac{\partial L}{\partial \dot r}$.

[1] “Mathematical methods of classical mechanics“ V.I. Arnold, §15 Cor.2.

$\endgroup$
  • $\begingroup$ FWIW, the very same Lagrangian issue in pppqqq's last example is precisely asked e.g. here & here. $\endgroup$ – Qmechanic Dec 26 '18 at 15:24
6
$\begingroup$

The canonical (Hamiltonian) formalism offers one of the main paths for quantizing gravity. General Relativity can be expressed in terms of the ADM 3+1 decomposition of spacetime:

http://en.wikipedia.org/wiki/ADM_formalism

And Hamiltonian's underlie quantum mechanics:

http://en.wikipedia.org/wiki/Hamiltonian_(quantum_mechanics)

Not only does this provide an elusive link between otherwise fundamentally incompatible theories (quantum field theory and general relativity), but in the Hamiltonian formalism of GR it is possible to solve problems numerically which are otherwise extremely difficult or impossible via the standard Einstein Field Equations.

By the way, the Lagrangian (and Lagrange density) is physical in general relativity, since one can derive the Einstein Field Equations directly from the Einstein-Hilbert action. This minimization of action is also the foundation of the path integral approach to quantum field theory:

http://en.wikipedia.org/wiki/Path_integral_formulation

The Feynman diagrams so useful in QFT derive directly from this, and of course string theory is a higher dimensional generalization of the path integral approach.

http://www.staff.science.uu.nl/~hooft101/lectures/stringnotes.pdf

$\endgroup$
6
$\begingroup$

Also, you can write Hamilton's equations of motion in sympletic form: $$ \dot\xi_i = \omega_{ij}\frac{\partial H}{\partial\xi_j} $$

Where $\xi_i$ are the coordinates in the phase space, that is, $\xi = (\mathbf q, \mathbf p)$. And, $\omega$ is the sympletic matrix: $$ \omega = \begin{bmatrix} 0 && -I_{n\times n} \\ I_{n\times n} && 0 \\ \end{bmatrix} $$

Where $I_{n\times n}$ is the identity matrix, with a system of $n$ spatial coordinates (and thus, $n$ speeds, and those, $2n$ amounts of phase space coordinates). Also, for an observable $G$, we have: $\dot G = \{G, H\}$ as you know. So, you can easily have the dynamics of an given observable $G$. All very nice and neat and general, but....

But... here is what I consider to be the most amazing part of hamiltonian mechanics: $$ X = x^i\partial_i = \{\xi^i, H\}\partial_i $$

Where $X$ is an hamiltonian vector field. Now, instead, we can generalize for an observable $G$, its vector field: $$ X_G = x^i_G\partial_i, \quad x^i_G = \{\xi^i, G\} = \frac{d\xi^i}{d\epsilon} $$

For any given parameter $\epsilon$ for observable $G$, generating an operator $X_G$. Its first order Taylor expansion: $$ \xi^i(\epsilon) - \xi^i(\epsilon_0) = (\epsilon - \epsilon_0)X_G\xi^i $$

Where operator $X_G$ is acting on $\xi^i$. We can solve the differential equation in successive infinitesimal transformations, arriving in the fundamental exponential limit, thus having the complete general solution of any hamiltonian system for any observable $G$: $$ \xi^i(\epsilon) = \exp\left(\Delta\epsilon X_G\right)\xi^i_0 $$

Do you understand the power of it?? Pointing out, again, here it is the solution of any hamiltonian system for any observable $G$ with parameter $\epsilon$ generated by operator $X_G$. If you want to analyse dynamics, then $\epsilon$ is the time, and $G$ is the hamiltonian, where $X_H$ defines the hamiltonian vector space. All hamiltonian systems have the same solution. The same solution!! So, lets solve for the dynamics (ie, where $\epsilon$ is time): $$ \xi^i(t) = \exp\left(\Delta t\frac{d}{dt}\right)\xi^i_0 $$

So, as you can see, pretty nice. Lagrange mechanics gives you nice unified equations of motion. Hamiltonian mechanics gives nice phase-space unified solutions for the equations of motion. And also gives you the possibility to get an associated operator, and a coordinate-independent sympletic-geometrical interpretation. The former is crucial in quantum mechanics, the later is crucial in dynamical systems.

$\endgroup$
  • $\begingroup$ Hm.. this is funny ... I think this is connected to my and other answers, extremely interesting angle, thank you ! $\endgroup$ – jhegedus May 16 at 6:29
  • $\begingroup$ Wait a sec, the best part of this answer is the name of the person who wrote this answer :) LOOOOL :):) $\endgroup$ – jhegedus May 26 at 23:09
4
$\begingroup$

The Hamiltonian can be used to describe an evolution of the "density in phase" of a system of N bodies. The density in phase is a conserved quantity for a system in equilibrium by Liouville's Theorem. The position and momenta can describe any general intensive parameter. Gibbs used this approach to derive statistical mechanics.

This approach of the concept of the evolution of a probability density function can be used in many other applications. My current research applies this to state space control theory, economic analysis, and assessing radiation damage in cells. So while it is a little convoluted, it is extremely useful. It goes hand in hand with entropy maximization.

$\endgroup$
  • $\begingroup$ Sorry, please correct this answer, I don't want to down vote it but "The density in phase is a conserved quantity for a system in equilibrium by Liouville's Theorem." is meaningless. There is no such thing as "density", at least not in a discrete system. Every state is either occupied or not. Liouville's theorem says that the VOLUME is constant. I.e., the number of occupied points. This only matters for an ensemble of systems (in stat phys) - where the volume is more than "one". "Information does not get lost." This is what it means. $\endgroup$ – jhegedus May 16 at 6:34
  • $\begingroup$ "It goes hand in hand with entropy maximization" => I would be extremely careful with saying stuff like this. "Boltzmann's reason powering the suicide were intellectually driven frustrations." It is still a mystery. claesjohnson.blogspot.com/2019/02/boltzmann-175.html The answer looks VERY shaky. $\endgroup$ – jhegedus May 16 at 6:43
3
$\begingroup$

One benefit of the Hamiltonian is the direct expression of Noether's theorem. Noether's theorem says symmetry's lead to conserved quantities.

One way to understand Noether's theorem is that a system with a symmetry has an associated ignorable coordinate in the Lagrangian. For example a system with rotational symmetry can be expressed in coordinates where the angle of rotation $\phi$ doesn't appear in the Lagrangian.

$$ \frac{\partial L}{\partial \phi} = 0 \implies \frac{d}{dt} \left( \frac{\partial L}{\partial \dot{\phi}} \right) = \frac{d p_\phi}{dt} = 0 $$

So the $\phi$ component of the momentum is conserved.

The Hamiltonian approach is especially useful in numerical methods. Notice how one of the Hamilton evolution equations tells us about changes in momentum.

$$\frac{d p_i}{dt} = -\frac{\partial H}{\partial q^i}, \quad \frac{d q^i}{dt} = \frac{\partial H}{\partial p_i}$$

In a system with conserved canonical momenta Hamilton's equations will explicitly demand conservation. In many cases the Hamiltonian, itself, is a conserved quantity (like energy). Finding numerical solutions to Hamilton's equations instead of Newton's second law will result in greater stability of the numerical solutions. There is a whole class of differential equation solving methods, symplectic integrators, that use this feature.

If you numerically evolve an orbital problem directly from $\vec{F}=m\ddot{\vec{x}}$, numerical error will build up rapidly and the orbit will diverge from the true solution. One way to see this is to compute the energy and angular momentum as functions of time ($E(t)$, $\ell(t)$) from position solutions $r(t)$, $\theta(t)$, $\phi(t)$. You'll find that $E(t)$ and $\ell(t)$ become significantly different from the starting values, and keep getting worse.

When working with Hamilton's equations, numerical error will affect your calculations, but $\ell()$ will be exactly the same at every step and $E(t)$ will be more stable as it is a function of the stable $p$'s in addition to $q$'s. The position coordinates will still have numerical error. But because $E$ and $\ell$ are stable, the coordinates will wobble around the true values rather than diverging.

$\endgroup$
2
$\begingroup$

Extremely brief and unmentioned answer: momentum and position in quantum mechanics (QM) form a representation of the Heisenberg algebra in terms of unitary operators. In Newtonian mechanics (NM) there is no visible underlying algebraic structure, but in Hamiltonian mechanics (HM) momentum and position also form a representation of the Heisenberg algebra, this time in terms of real fuctions. From this group-theoretical point of view, HM and QM are almost indistinguishable, whilst QM looks like magic compared to NM.

$\endgroup$
  • $\begingroup$ Unitary operators: preserving norm of wavefunction. Omg, representation. My group theory stops at: representation is a "matrix". Say, some rotation matrix, for some rotation group. Position and momentum, are not representations. The operators themselves are. If I take the matrix analogy. Position and momentum are just simply the "spaces" where the operators/matrices operate. My group theory is 18 years old, so I might be a bit wrong here. $\endgroup$ – jhegedus May 6 at 1:42
  • $\begingroup$ Interesting, today I just jumped into my mind the concept of irreducibility. I think it relates to transforming some matrix into block diagonal form. Very funnilly this also leads to the concept of independence as in de-coupling (my group theory is very old... but this block diagonal thing gives me a lead). $\endgroup$ – jhegedus May 16 at 6:26
  • $\begingroup$ Uuuups, yes ... en.wikipedia.org/wiki/Irreducible_representation ... the main importance of group theory is exactly this ... to describe systems in terms of decoupled groups... $\endgroup$ – jhegedus May 16 at 6:27
1
$\begingroup$

The classic problem of mechanics is to solve the equations of motion for a given Lagrangian or Hamiltonian system. In this case it is just a matter of choice whether to use the Hamilton- or Lagrange formalism to do this. Once the solution is found, everything there is to know about that specific system is contained in it.

But how about if one wants to ask more fundamental questions of whether there are properties of physical systems that are not specific to the particular form of a Hamiltonian/Lagrangian but are rather inherent to all systems. In order to answer this, one needs to unravel the mathematical structure that is generic to all physical systems. It is exactly then when the Hamilton formulation differentiates from the Lagrangian formulation: The generic structure that is underlying Hamiltonian systems goes under the name “symplectic manifolds” and it turns out that its mathematics is so rich that it is of great interest to mathematics up to this date.

The most prominent example of a generic property of Hamiltonian systems that is not related to the specific form of a Hamiltonian is the Liouville theorem which states that phase space is preserved with time. Intuitively this means that information never gets lost during the life of the system.

Studying Hamiltonian dynamics/symplectic manifolds becomes particularly useful when space time is not Euclidian. For example symplectic manifolds and hence Hamilton dynamics do not exist on a sphere $S^{2n}$ for n>1. So it is these types of questions that can very naturally be studied in the symplectic manifold/Hamiltonian setting rather than the Lagrangian formalism.

$\endgroup$
  • $\begingroup$ I like that you also bring up the information angle. The story goes much deeper than that. Consider independence. Independent systems. Decoupling. The entropy is additive. The energy is additive. Energy is simple a measure of entropy. The equation of motions are independent. IMHO this goes both ways. Also, I suspect, that this gives meaning to "dimensions". Generalized coordinates. Which take constraints into account. They (can) decouple the energy dependence. So H(x,y)=H(x)+H(y) becomes possible to write. Due to the "concept" that the total energy is the sum of the component energies. $\endgroup$ – jhegedus May 16 at 6:13
  • $\begingroup$ I came to this conclusion after learning "ML" for many hours, 3 years + 2 or so... and also have a physics PhD, with focus on theoretical condensed matter physics. Funnily ML and cond mat phys fit splendidly smoothly together. Like spin glasses for DNNs and such. So I was wondering why is so extremely big deal that "stuff" has to be independent in "ML" (see graphical models and their friends), nevertheless, the concept of independence is central in ML. Finding independent "coordinate system". It makes it possible to write the entropy as a sum. Then I thought about stat phys, entropy $\endgroup$ – jhegedus May 16 at 6:17
  • $\begingroup$ free energy and how thermodynamics is derived from simple principles, ergodicity, phase space, entropy and such... I came to the realisation that the existence of Hamiltonian formalism and independence in "ML" has the same underlying reason, namely, reducing the Kolmogorov complexity of the algorithm that describes the "theory" in physics, or the "model" in ML. Also, the prior (in both disciplines) is exponentially getting sparser with the number of independent degrees of freedom. Also, once independence is maximized, it becomes easier to "do" unsupervised learning (I forgot why ... :) ). $\endgroup$ – jhegedus May 16 at 6:23
0
$\begingroup$

The following answer is a bit "intuitive" but hopefully still mostly correct, or at least thought provoking. Sorry for the lack of rigor. I plan to write down these thoughts one day into some nice blog post, this is just a rough sketch.

I am not sure but the biggest point in the concept of "Hamiltonian" is that two independent systems Energy are additive.

Non interacting systems can be described by H1+H2.

I searched this page and this has not been described.

Why is this additivity such a big deal ?

Let's take a harmonic oscillators.

The phase space is a circle's circumference.

Energy is proportional the radius of the circle.

So the circumference.

So the number of microstates.

So S=-log(E)*c.

So why is this a big deal ?

Because if we take two harmonic oscillators, then the entropy becomes additive (extensive).

So why is this such a big deal ?

Probability. Independent systems log likelihoods are additive.

So, the physical independence and the probabilistica independence in this case are the same.

So statistical physics becomes possible to "be done".

This is a different take on the accepted answer's statement. From an information theoretic point of view.

Why is independence such a big deal ?

The Kolmogorov complexity of the algorithms that describe the phase space, or even the motion are additive. So it is optimal. In the sense of Occam's razor.

Hence the Hamiltonian formalism is the most optimal way to create theories that describe independent systems.

From this point of view it is intuitive to see that perturbation theory "works".

If the change in energy of one subsystem is small (weak perturbation) then the phase space does not become much larger, so the information to be stored to describe the perturbed system is not much more because the size of the phase space does not change much.

So this information theoretic approach gives an intuitive explanation why perturbation theory "works".

Also, E=mc^2 follows from this (up to a constant). E=mc^2 is simply expresses that if one oscillator disappears then it's phase space disappears too, and the energy is transferred to the other oscillator, so the information is conserved. E=mc^2 is "simply" about conservation of information. Without the concept of Hamiltonian this equation and the corresponding conservation of information would not exist.

So the Hamilton equation is important because it makes it possible to treat independent systems independent in the information theoretic (from which follows probabilistic) frameworks, as this was hinted at the first point in the first answer. Statistical mechanics is based on this. Also, thermodynamics would not exist with the concept of Energy. Since independent systems are described by their energy which is extensive, additive.

Interestingly, all extensive variables in thermodynamics are related to changed of the phase space. Volume grows, volume related phase space changes, kinetic energy decreases (momentum related phase space decreases), in adiabatic systems, so that the total information content stays constant (and consequently the entropy).

So without Energy there is no Entropy, no information, no phase space, no E=mc^2.

Why ? Without Energy there is no independence between isolated systems.

Why is that wrong ? Theories (algorithms) that describe independent systems have additive Kolmogorov complexity. Without the concept of Energy theories would not have this property hence would not obey Occam's razor, hence would be unnecessarily more complex than needed. Would be less correct.

In the framework of Solomonoff's theory this statement can be justified.

$\endgroup$
  • $\begingroup$ You make good points, but aren't the Lagrangians of independent systems also additive? $\endgroup$ – Javier May 5 at 13:48
  • $\begingroup$ No. Why would they be ? You can add them together but that has no relationship to the phase space / entropy / information theory / thermodynamics / conservation laws / nothing. HOWEVER ! Lagrangian approach has one other strength : variational principle. youtube.com/watch?v=KdFo6eoW9x4 This is fantastic too. $\endgroup$ – jhegedus May 5 at 15:19
  • $\begingroup$ and, as an additional touch : Virial theorem. en.wikipedia.org/wiki/Virial_theorem physics.stackexchange.com/questions/32896/… Virial theorem follows from least action principle. So this can give some food for inspiration - about which I will need to think myself ! $\endgroup$ – jhegedus May 5 at 15:22
  • $\begingroup$ I would even contemplate that this is a sort of dualism. Lagrangian is time independent, while Hamiltonian is space independent approach. In other words, Lagrangian is based on space variation but not on time variation. Hamiltonian is based on time variation (time evolution) but not on space variation (does not need to have derivative by space). $\endgroup$ – jhegedus May 5 at 15:29
  • $\begingroup$ discourse.mc-stan.org/t/… is discussed here too, hopefully :) $\endgroup$ – jhegedus May 16 at 17:23
0
$\begingroup$

In algorithmic information theory (Leibniz, Kolmogorov, Chaitin) , there is a concept, " elegant programs" . These are minimal programs that can generate a given binary sequence. We can make an analogy with physics, or any other axiomatic theory (this analogy has been studied by Chaitin). The Lagrangian, Hamiltonian formalism (with the min action principle ) represent a minimal mathematical framework that can explain a lot of experimental data, from all domains of physics, from QFT to GR. Unfortunately, in algorithmic information theory, proving that a program is " elegant" is not a trivial problem. Following this analogy, whether the Lagrangian/Hamiltonian formalism is the best possible, this is not a trivial problem. To answer your question, the point is "elegance ", or minimality (in mathematical axioms/principles needed ).

$\endgroup$
  • $\begingroup$ Exactly ... my point was this. $\endgroup$ – jhegedus May 5 at 21:09
  • $\begingroup$ But... there is no free lunch... so ... that is your point, I guess. $\endgroup$ – jhegedus May 5 at 21:11
  • $\begingroup$ AFAIK this gives a nice def. for elegance : en.wikipedia.org/wiki/… $\endgroup$ – jhegedus May 5 at 21:12

protected by Qmechanic Sep 24 '16 at 17:42

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.