# Uncertainty Principle for Information?

I'm not familiar (yet) on how Information theory can be emerged/used in QM/QFT but I was thinking about this question:

While we have Heisenberg uncertainty principle on measuring coupled observables, can we express it using somehow a more fundamental/abstract concept like Information uncertainty (especially that we have Information conservation principle like for energy and momentum, and associated symmetry, i.e CPT)? in sense that because there is always some details that we can't measure/know, can we express that as some information uncertainty?

And despite of the answer please explain why there is or there isn't connection/relation between the two concepts.

-
Information theory has at least two general approaches, Shannon's, and Kolmogorov's. Also, since quantum processes are reversible, that makes information a conserved quantity. –  Mike Dunlavey Nov 1 '12 at 12:18
@Mike: do you pointing to that Info. conservation is due to reversibility, not due to CPT? aren't they connected at the end? –  TMS Nov 1 '12 at 12:24
Under unitary operations, information (entropy) is conserved. However, the uncertainty principle is about the quantum measurement process, which manifestly does not conserve information because quantum states change discontinuously and irreversibly. –  Mark Mitchison Nov 1 '12 at 12:41
@Mark: You said it more precisely than I did. –  Mike Dunlavey Nov 1 '12 at 12:54

Lubos' answer is correct: information is not an observable so does not have fluctuations in the sense that could enter an uncertainty relation. However, there does exist a relationship between 'information' and the uncertainty principle, although not of the type that it seems the OP expects.

First of all, note that 'information conservation' could never be an explanation for the uncertainty principle. Information is not a conserved quantity in quantum mechanics, since measurements are part of the formalism. Measurements, by definition, produce a discontinuous change in the information content of a system with respect to an observer. It is important to remember that, despite its current fashionable status as a paradigm for understanding physics, information is still not a physical property of a system. Rather, it is a property of the relationship between an observer and a system. The only subtlety is that quantum mechanics places a fundamental restriction on the amount of information that can be gained by any observer.

To understand how to make this restriction quantitative, you will need to learn a bit of quantum estimation theory. I am not gonna derive it all here; you can find details in reviews such as, for example, this paper. The basic idea is that if you want to estimate some parameter $\lambda$ on which a state depends, which may or may not be an "observable" in the traditional sense, your precision will be limited by the Cramer-Rao bound: $$\mathrm{Var}(\lambda) \geq \frac{1}{M F(\lambda)},$$ where $\mathrm{Var}(\lambda)$ is the variance of the distribution of measurement outcomes, $M$ is the number of measurements and $F(\lambda)$ is the so-called Fisher Information. This is a result from classical information theory.

Given a system and a parameter to be estimated, the Fisher Information generally depends on the choice of measurements. In the quantum case, one can do even better and show that the Fisher Information is bounded from above by the Quantum Fisher Information $H(\lambda)$,so the quantum Cramer-Rao bound reads $$\mathrm{Var}(\lambda) \geq \frac{1}{M H(\lambda)}.$$ The quantum Fisher information gives the absolute upper bound on the amount of information that an observer can gain about the parameter $\lambda$ by measuring the system. It is the Fisher information corresponding to the optimal measurement basis.

How does this relate to the uncertainty principle? Specialise to the particular case of a system in a pure state, where the parameter dependence is produced by the unitary transformation $$|\psi(\lambda)\rangle = U_{\lambda}|\psi(0)\rangle,$$ where $$U_{\lambda} = e^{-\mathrm{i} \lambda G},$$ and $G$ is the hermitian generator of the unitary transformation. This includes scenarios such as energy-time uncertainty, where $G = \hat{H}$ is the Hamiltonian (generator of time translations) and $\lambda = t$ is the waiting time after initial preparation of the state $|\psi(0)\rangle$ (I set $\hbar = 1$). Then you can derive the following inequality from the quantum Cramer-Rao bound: $$\mathrm{Var}(\lambda) \langle \psi(0)| G^2 |\psi(0)\rangle \geq \frac{1}{4 M},$$ which is exactly an uncertainty relation. Note that this example is slightly artificial: the uncertainty relations are more general than this scenario. However, hopefully this example gives you a flavour of how uncertainty relations can be linked to concepts from information theory. (It also shows that energy-time uncertainty relations don't require a lot of hand-waving to derive, as some people seem to believe.)

Another subtle connection that is worth mentioning is a very deep fact about quantum mechanics: "information gain implies disturbance". This means that it is impossible to gain some information about a system without disturbing it. The more information gained, the greater the disturbance. See this paper for more info. If you take the information-disturbance trade-off as a fundamental principle for quantum mechanics, such as in this recent paper, then you have a heuristic way of understanding the physical origin of the uncertainty principle.

-
the OP states that information is conserved in the sense that all evolution is unitary. He doesn't refer necessarily to usable information, but information in the quantum state of the universe (microstate) –  lurscher Nov 1 '12 at 16:23
@lurscher Looks like you are putting words in the OP's mouth to me. In any case, ascribing a pure, unitarily-evolving state to the universe as you implicitly have done in your comment is a metaphysical statement of questionable physical content. –  Mark Mitchison Nov 1 '12 at 17:01
Have tried to edit but not easy on a phone :). Let me just add that I'm sorry if I did misunderstand the op's intention but the concept of 'information content' of the universe is not even meaningful if you accept my proposition that information is not entirely a physical property. –  Mark Mitchison Nov 1 '12 at 17:09
@Mark: thx for your satisfactory answer. –  TMS Nov 1 '12 at 19:10

The Heisenberg uncertainty relationship holds for observables – something that may be measured by an apparatus and is, according to the universal rules of quantum mechanics, represented by a linear operator on the Hilbert space.

Information isn't an observable – because of both reasons (it can't be measured by a gadget; and it isn't a linear operator although "the logarithm of the density matrix" $-\ln\rho$ comes pretty close to that description) – so it can't enter the Heisenberg uncertainty relationship. Incidentally, "the amount of information" is somewhat uncertain or ill-defined even in classical physics – so this uncertainty has nothing to do with quantum mechanics.

The latter point may be explained by one more simple argument. Note that the Heisenberg inequality has $\hbar$, the reduced Planck constant on the right hand side. It's a quantity that is sent to zero in the classical limit. From our, large observers' viewpoint, it is a small number which is why the uncertainty relation is a "minor effect" at the macroscopic scale.

But in the usual units, $\hbar$ is dimensionful (units of action, i.e. of energy times time) so it's clear that the observables on the left hand side must be dimensionful, too. Position and momentum are; information (the number of bits, a dimensionless number) isn't. So by dimensional analysis, the uncertainty of information has nothing to do with quantum mechanics.

The very claim that information is "more fundamental" is somewhat speculative and loaded. One may create such "priorities" and yes, indeed, what we know is the information. However, as long as we express what we know in some tangible way, we have to use particular observables such as $x,p$ or the angular momentum etc. Only when we get to this level – whether you call it fundamental or not, but you should – we may talk about well-defined principles such as the uncertainty principle. It's a very fundamental principle; it just disagrees with the (empirically unkjustified) philosophy that the information is more fundamental than particular observables.

-
Thx for the answer. First of all I can't agree with your assumption that uncertainty of information should be of the form Heisenberg uncertainty, not speaking of your try for dimensional analysis of that. Secondly as it's known Uncertainty is due to quantum nature, not due to measuring process it self, thus uncertainty should has somehow different meaning here. Finlay, I meant by fundamental, more abstract, my fault, and my approach was that because there is always something that we can't know exactly, may be this can be expressed as information luck? –  TMS Nov 1 '12 at 12:13
Dear TMS, what the uncertainty exactly means is subtle and one must be careful about it and it seems you are trying to be careful but it doesn't quite look you are succeeding. ;-) To define information in QM, e.g. by $-{\rm Tr}\,\rho\ln\rho$ from a density matrix, is a bit nontrivial. The Gaussian wave functions that minimize $\Delta x \cdot \Delta p$ are pure states which means that they have "zero information". But pure states that don't minimize the uncertainty product have "zero information", too. So your intuition that the absence of information "is" the uncertainty isn't really right. –  Luboš Motl Nov 1 '12 at 14:24
In some sense, you are slightly misinterpreting the term "information". You seem to think that information is the same thing as accuracy - most things you would call "lots of information" is about "saying values of observables with a good accuracy". But that's not how it works, especially not in QM. Exactly because of the uncertainty principle, the accuracy of $x,p$ can't exceeed certain (correlated) bound and there can be no information beyond it, not even in principle. Instead, most information is about knowing "at least something" about many observables, e.g. coordinates of many atoms. –  Luboš Motl Nov 1 '12 at 14:26
Let me say it again, slightly differently. The uncertainty principle says that there can exist no information about the "very accurate" value of $x$ and "very accurate" value of $p$, not even in principle. Because this information can't exist (because the states with very precise $x,p$ don't exist, either), it can't be made absent, either. ;-) But information may only express what you need to know to distinguish which option among "a priori possible options/states" is realized. If something is impossible in general, saying that it's not realized in a situation provides you with NO information. –  Luboš Motl Nov 1 '12 at 14:29

You are looking for the Hirschman Beckner uncertainty principle, described on the Wikipedia page here. While Hirschman was first, I learned about it from reading Everett's thesis on the many worlds interpretation of quantum mechanics, which is an attempt to reformulate quantum mechanics using Shannon information tools.

The statement of the principle is that

$$H(x) + H(p) \ge \ln(e\pi)$$

And that this inequality is saturated exactly for Gaussian wavepackets. It was conjectured by Hirschman, and proven by Beckner in 1975. Everett's thesis appears after Hirschman's paper, but it is unclear to me which way the plagiarism goes, if any, since this is very likely to be a simultaneous discovery.

Everett's argument shows that the Gaussians are local minima for the sum of $H(x)$ and $H(p)$ and gives strong reasons to believe it is a global minimum. This belief is justified by the rigorous proof. The information theoretic formulation is more powerful than the formulation in terms of variance, since if the x distribution is a sum of many narrow far-apart spikes, the momentum uncertainty is as the width of the spikes, not as the distance between them, even though the total variance is as the distance between them, not as the width of the spikes themselves.

-
Momentum uncertainty as the width of spikes... interesting approach, thx for inf. –  TMS Nov 3 '12 at 9:46