3
$\begingroup$

I've only ever heard about information theory being used in stuff related to probability distributions, which makes sense because information and entropy are related. However, I'm having trouble finding resources on a direct relationship between newtonian mechanics and information dynamics, which is weird because most of statistical mechanics comes from emergent properties of "bouncy balls" that follow the axioms of newtonian motion.

Is there a formalism that relates information theory to classical mechanics directly? I doubt there is a way to do that, but there should at least be some formulation that can equate $F=ma$ to some other variable which comes from Shannon's work. Or talk about the average force of each particle in the system using entropy.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Information theory needs many microstates, so you can't relate just 1 body with entropy, but you can relate thermodynamics (particle ensemble) or statistical physics if you like,- with information theory by a Boltzmann law : $S=k_{\text{B}}\ln W$. $\endgroup$
    – Agnius Vasiliauskas
    Commented May 16, 2023 at 8:29
  • 1
    $\begingroup$ The thing is, those microstates arise naturally from classical mechanics. I've been trying to get at the bottom of this and found a paper where they use a so-called "information metric", which supposedly establishes that relationship by keeping the position of the particle probabilistic, but that just sounds artificial. $\endgroup$
    – user367128
    Commented May 17, 2023 at 22:16
  • 2
    $\begingroup$ Yes, it is. Probabilities arise natural in the statistical physics, where you have ensemble of particles. If you take just 1 rigid body or particle , then you need to inject some uncertainties a-priory into the model to get the microstates. But, this will be the mix of classical and quantum mechanics, at least in terms of Heisenberg uncertainties. So plain classical physics is gone. Hence, that's why I've said that $F=ma$, nor any kinematic laws will not help you to arrive at information theory. $\endgroup$
    – Agnius Vasiliauskas
    Commented May 18, 2023 at 6:31
  • $\begingroup$ See, the real problem is that, for example, energy is indeed related to newtonian mechanics, you can calculate it easily, AND you can take the average energy of the particles in a gas to get the entropy, without actually talking about the microstates. What I'm trying to get at is that, surely if we assign a random probability distribution associated with the movement of each particle, we could factor out the average force of a particle and express it in terms of the entropy of the whole system. $\endgroup$
    – user367128
    Commented May 18, 2023 at 16:39
  • 1
    $\begingroup$ You need to be careful with introducing "information" into physics. Proper physics is always about energy, momentum, angular momentum and charges (this drops out of relativity and Noether's theorem). The only place where information and physics touch is in the rare case of systems that obey the ergodic hypothesis, which most systems do not. $\endgroup$
    – FlatterMann
    Commented May 18, 2023 at 21:01

2 Answers 2

Reset to default
0
$\begingroup$

I'm not sure I'm understanding your question completely, but I'll say this.

Newtonian mechanics uses its postulates to fully solve a problem, one where we have all the information (and we're in the classical limit).

If you try to use it to solve a different problem, one where you don't have all the information, it will fail. But you can make a few assumptions (like ergodicity) and then apply information theory in combination with Newton, and then you'll solve the problem (though this solution will be probabilistic of course).

Information theory is purely mathematical, Newtonian is physical. You can't get one from the other. Rather, you insert Newton into Information to get Statistical mechanics. (You can insert Quantum mechanics instead to get a different version!)

I think that you can take Information theory and tune one of its parameters, to the point that all probabilities become either 1 or 0, so it makes perfect predictions based on total information.

Then you'd be back at the first case where Newton works (and information theory becomes trivial and adds nothing to the theory)

Improve this answer
$\endgroup$
0
$\begingroup$

Chaos really plays an essential role here to obtain statistical mechanics out of Newtonian mechanics. As a matter of fact, if you have an integrable system, it is known that the system does not thermalize, i.e. statistical mechanics fails (at least in the naive sense).

When you do have a chaotic Newtonian-mechanical system (which is the vast majority of systems), if you only have a finite precision about the initial state (which is a very natural assumption) the uncertainty that comes from that initial precision grows exponentially as a function of time. If you assume that the likelihood (aka probability) of the system's true initial state being uniformly likely over all possible states within your precision, the probability mass quickly dissipates through the state space (c.f. Liouville's theorem), which justifies the usage of probability distributions such as canonical and microcanonical ensembles.

In some sense, you can say that the chaotic nature of the Newtonian mechanical system naturally defines a probability distribution (like the canonical ensemble), and then you can apply information theory to that distribution. The most famous one being Shannon's entropy matching Boltzmann's entropy for the classical Gibbs distribution.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.