# How is information related to energy in physics?

I recently attended a talk by Dr. Ravi Gomatam on 'quantum reality', where the speaker suggested, that conservation of energy is not a fundamental law, and is conditional, but the conservation of information is fundamental. What exactly is the meaning of information? Can it be quantified? How is it related to energy?

-
possible duplicate of Are information conservation and energy conservation related? –  Qmechanic Mar 8 '12 at 15:22
Thanks for making me aware of this guy (Dr. Gomatam). –  WetSavannaAnimal aka Rod Vance Jul 17 '13 at 2:42

If one measures lack of information by the entropy (as usual in information theory), and equates it with the entropy in thermodynamics then the laws of thermodynamics say just the opposite: In an isolated system, energy is conserved wheras entropy can be created, i.e., information can get lost.

The main stream view in physics (aside from speculations about future physics not yet checkable by experiment) is that on the most fundamental level energy is conserved (being a consequence of the translation symmetry of the universe), while entropy is a concept of statistical mechnaics that is applicable only to macroscopic bodies and constitutes an approximation, though at the huma scale a very good one.

-
What you have said, is just a statistical definition of information. Can physical information be defined is some other way ( a quantum mechanical definition?) –  user7757 Mar 8 '12 at 15:41
Information is a statistical concept, also in telecommunication engineering, say. It captures the scientific aspect of information, though not its subjective value for human beings. Maybe you can ask morespecifically after having read en.wikipedia.org/wiki/Physical_information –  Arnold Neumaier Mar 8 '12 at 16:00
Ramanujan, do you happen to know any online links to explain what Dr Gomatam means by the assertion that conservation of energy (and I'll assume this really means mass-energy) is not fundamental? That alone is a very unusual assertion, so it's not clear exactly what he intended. I looked at his home page, but nothing looked promising based on the titles. –  Terry Bollinger Mar 8 '12 at 23:41
@ramanujan : it would be interesting to know about the "conditions" at which the speaker says energy is conserved or for that matter not conserved. I have recently asked a similar question in physics.SE, and got an answer that conservation of energy is fundamental which comes from Noether's Theorem, basically because of symmetry in space-time. –  Vineet Menon Mar 9 '12 at 5:52
@TerryHey, sorry if you have already seen this link. bvinst.edu/faculty/~gomatam.htm There is a link under lectures 'quantum reality - Why physicists dont understand it', it seems he gave the same lecture at my institute –  user7757 Mar 9 '12 at 12:11

In contrast to @ArnoldNeumaier, I'd argue that the information content of the World could be constant: it almost certainly can't get smaller and how it and if it gets bigger depends on the resolution of questions about the correct interpretation of what exactly happens when one makes a quantum measurement. I'll leave the latter (resolution of quantum interpretation) aside, and instead discuss situations wherein information is indeed constant. See here for definition of "information": the information in a thing is essentially the size in bits of the smallest document one can write and still uniquely define that thing. For the special case of a statistically independent string of symbols, the Shannon information is the mean of the negative logarithms of their probabilities $p_j$ of appearance in an infinite string:

$H=-\sum_j p_j \log p_j$

If the base of the logarithm is 2, H is in bits. How this relates to the smallest defining document for the string is defined in Shannon's noiseless coding theorem.

In the MaxEnt interpretation of the second law of thermodynamics, pioneered by E. T. Jaynes (also of the Jaynes-Cumming model for two level atom with one electromagnetic field mode interaction fame), the wonted "observable" or "experimental" entropy $S_{exp}$ (this is what the Boltzmann H formula yields) of a system comprises what I would call the true Shannon information, or Kolmogorov complexity, $S_{Sha}$, plus the mutual information $M$ between the unknown states of distinguishable subsystems. In a gas, $M$ measures the predictability of states of particles conditioned on knowledge about the states of other particles, i.e. is is a logarithmic measure of statistical correlation between particles:

$S_{Exp} = S_{Sha} + M$ (see this reference, as well as many other works by E. T. Jaynes on this subject)

$S_{Sha}$ is the minimum information in bits needed to describe a system, and is constant because the basic laws of physics are reversible: the World, therefore, has to "remember" how to undo any evolution of its state. $S_{Sha}$ cannot in general be measured and indeed, even given a full description of a system state, $S_{Sha}$ is not computable (i.e. one cannot compute the maximum reversible compression of that description). The Gibbs entropy formula calculates $S_{Sha}$ where the joint probability density function for the system state is known.

The experimental (Boltzmann) entropy stays constant in a reversible process, and rises in a non-reversible one. Jaynes's "proof" of the second law assumes that a system begins with all its subsystems uncorrelated, and therefore $S_{Sha} = S_{exp}$. In this assumed state, the subsystems are all perfectly statistically independent. After an irreversible change (e.g. a gas is allowed to expand into a bigger container by opening a tap, the particles are now subtly correlated, so that their mutual information $M > 0$. Therefore one can see that the observable entropy $S_{exp}$ must rise. This ends Jaynes's proof.

See also this answer, for an excellent description of entropy changes an irreversible change. The question is also relevant to you.

Energy is almost unrelated to information, however, there is a lower limit on the work must do to "forget" information in a non reversible algorithm: this is the Landauer limit and arises to uphold the second law of thermodynamics simply because the any information must be encoded in a physical system's state: there is no other "ink" to write in in the material world. Therefore, if we swipe computer memory, the Kolmogorov complexity of the former memory state must get pushed into the state of the surrounding World.

Afterword: I should declare bias by saying I subscribe to many of the ideas of the MaxEnt interpretation, but disagree with Jaynes's "proof" of the second law. There is no problem with Jayne's logic but (Author's i.e. My Opinion): after an irreversible change, the system's substates are correlated and one has to describe how they become uncorrelated again before one can apply Jaynes's argument again. So, sadly, I don't think we have a proof of the second law here.

-
I think you've misunderstood Jaynes' proof. Jaynes doesn't say the system's components become uncorrelated, he says that (some of) the correlations become irrelevant for making predictions about the system's future behaviour, so you can safely forget about them. Thus we pretend that the system's components have become uncorrelated, even though we know they haven't really, because this allows us to do calculations that would be completely intractable otherwise. –  Nathaniel Jul 16 '13 at 6:44
He makes the argument much more clearly in section 4 of this paper. –  Nathaniel Jul 16 '13 at 6:47
@Nathaniel I'm trying not to get too far off the track here. I am quite familiar with the argument you cite above, but I still think it's begging the question (by brining in further assumptions about what is and what is not "relevant" mutual information). I am not convinced there is an argument that fully resolves the Loschmidt paradox aside from saying that the second law is about boundary conditions of the universe. I understand that not everyone agrees on this point. Moreover, please understand that I would not call ANY of Jayne's assumptions unreasonable. –  WetSavannaAnimal aka Rod Vance Jul 16 '13 at 7:01
Jaynes argument for the second law is very explicitly based on an empirical fact. That fact is that we, as experimenters and engineers, are able to directly influence the initial conditions of an experiment, but we can't affect the final conditions except indirectly, by changing the initial conditions. Jaynes' argument says that given this asymmetry, the second law follows. However, this empirical fact in itself is then in need of an explanation, which Jaynes' argument can't help us with, and that's probably where you have to start thinking about the boundary conditions of the universe. –  Nathaniel Jul 16 '13 at 16:55
@Nathaniel I'm glad we seem to agree then. One sometimes reads people citing Jaynes's work as a "mathematical proof" for the second law, independently of experiment and that it explains the arrow of time. I don't believe that Jaynes himself ever claimed this status for his work - indeed he seems often to be "going the other way", i.e. beginning with physical reality and experiment and calling on these to shed light on the philosophical foundations of chance and randomness. This is why I baulk at calling his work a "proof" of the second law. –  WetSavannaAnimal aka Rod Vance Jul 17 '13 at 4:10