Hamiltonian Monte Carlo: Kinetic and Potential energies In HMC (good intro here) the Hamiltonian is defined as
\begin{align}
 H(p,\theta) &= -\log p(q,\theta) 
\\
&= -\log p(q) - \log p(\theta).
\end{align}
Here $p$ is the momentum coordinate and $\theta$ is the position coordinate.
My question is: why do we define the negative log joint as the Hamiltonian (total energy of the system)? Why do we interpret $-\log p(q)$ as the kinetic energy and $-\log p(\theta)$ as the potential energy?
I see why the kinetic energy should be a function of the momentum and potential energy is a function of the position. But why these functions in particular? In physics, kinetic energy is usually defined as $K = \frac12 mv^2 = \frac12 p^2/m$. I thought that we view $-p(\theta)$ as a potential field (done so that "kicking around a hockey puck" will lead us to maxima of the distribution), so why do we use $-\log p(\theta)$ for potential energy?
 A: In classical statistical physics, if using the Boltzmann distribution (canonical ensemble), the probability of a system to be found in a volume of phase space near momenta $p$ and coordinates $q$ is given by (in absence of magnetic field)
$$
w(p,q)\propto e^{-\beta H(p,q)}=e^{-\beta [K(p)+V(q)]},
$$
where $\beta=1/(k_B T)$ is the inverse temperature, $K(p)$ is the kinetic energy, and $V(q)$ is the potential energy.
Hamiltonian Monte Carlo (HMC)
MCMC using Hamiltonian dynamics by Neal is very elucidating regarding the relation between Hamiltonian Monte Carlo and physics (it can be also found here and cited among other books, if one folows the link given in the OP).
A bit of history
Historically HMC originates from molecular dynamics simulations of thermodynamic systems, which are obviously described by a Boltzmann distribution (the equation above), hence the terminology.
Maximum likelihood estimate (MLE) in statistics
In statistics one is often interested in maximizing a likelihood (MLE) (or a posteriori probability in a MAP estimate - a Bayesian equivalent of MLE), which can be viewed as a probability function of its parameters, $p(q)$. For mathematical and computational purposes one often chooses to maximize the log of this function (log-likelihood, $l(q)=\log w(q)$) which reaches maximum at the same values of parameters $q$). Note also, that in Markov chain Monte Carlo (MCMC) estimations one is usually unable to obtain the normalization constant, but only the shape of the probability distribution and the means of functions of interest.
Variable augmentation
In Hamiltonian monte Carlo one presents the probability function of interest as a Boltzmann distribution
$$w(q) \propto e^{-\beta V(q)},\beta V(q)=-l(q)=-\log w(q)$$
where the inverse temperature can always be taken to be $\beta=1$. One then performs variable augmentation by adding momenta corresponding to each of the components of $q$, and having distribution
$$w(p)\propto e^{-\beta K(p)},$$
where $K(p)$ is taken to be bilinear function of its variables
$$K(p)=\frac{1}{2}p^T\hat{M}^{-1}p$$ i.e., the momentum distribution is a multivariate Gaussian. If the "mass matrix" is diagonal, we simply have
$$K(p)=\sum_i\frac{p_i^2}{2m_i}.$$
The resulting system then resembles a system of interacting particles in a conservative potential, and inherits from it the well-known properties of energy conservation, phase volume conservation, time reversibility, etc. (See the Neal's chapter for the detailed discussion). One can then demonstrate that applying MCMC to this augmented system would produce the same results, as if we were sampling directly from the original distribution $w(q)$.
HMC
The simulation typically proceeds in the following steps:

*

*randomly drawing a momentum value from its normal/Gaussian distribution

*simulating Hamiltonian evolution of the system (usually numerically; e.g., using the leap-frog algorithm)

*accepting or rejecting the new state, as is usually done in the Metropolis-Hastings algorithm.

HMC is in fact a Metropolis-Hastings algorithm, but with a proposal density that is given by Hamiltonian dynamics of the augmented system. From the statistical point of view it has important advantages, such as not getting stuck near local maxima of the likelihood function (i.e., local potential minima), if the latter is multimodal (a common feature in mixture models, used for clustering). In physics language such stucking corresponds to ergodicity breaking, e.g., as in magnetic materials - in this sense HMC is said to preserve ergodicity.

References:
Radford Neal, MCMC using Hamiltonian dynamics
Michael Betancourt, A Conceptual Introduction to Hamiltonian Monte Carlo by Michael Betancourt
See also this post for a somewhat fine difference between Hamiltonian Monte Carlo and Metropolis-Hastings MCMC using Hamiltonian dynamics.
