# Is the density operator a mathematical convenience or a 'fundamental' aspect of quantum mechanics?

In quantum mechanics, one makes the distinction between mixed states and pure states. A classic example of a mixed state is a beam of photons in which 50% have spin in the positive $z$-direction and 50% have spin in the positive $x$-direction. Note that this is not the same as a beam of photons, 100% of which are in the state $$\frac{1}{\sqrt{2}}[\lvert z,+\rangle + \lvert x,+\rangle].$$ It seems, however, that at least in principle, we could describe this beam of particles as a single pure state in a 'very large' Hilbert space, namely the Hilbert space that is the tensor product of all the $\sim 10^{23}$ particles (I think that's the proper order of magnitude at least).

So then, is the density operator a mathematical convenience, or are there other aspects of quantum mechanics that truly require the density operator to be a 'fundamental' object of the theory?

(If what I mean by this is at all unclear, please let me know in the comments, and I will do my best to clarify.)

-
It is fundamental in the sense that it is required for representing states of tensor factor subsystems, but it may not be fundamental for an objective global perspective. However, there is no fundamental reason why the global state of the universe (if it exists) couldn't be more general than a pure state and instead be represented by a density operator. –  A.O.Tell Oct 30 '12 at 8:31
@A.O.Tell Can you elaborate on what exactly you mean by "representing states of tensor factor subsystems"? –  Jonathan Gleason Oct 30 '12 at 16:12
–  Arnold Neumaier Oct 31 '12 at 16:28

No macroscopic quantum system is described by a pure state. For example, notions like temperature or pressure, which apply to macroscopic systems do not even exist for systems described by a pure state. The description of macroscopic objects (discussed in statistical mechanics) is always in terms of a density matrix (or the essentially equivalent notion of a positive linear functional on a suitable $C^*$-algebra of observables.

However, what is considered as fundamental depends on the fundamentals upon which one erects QM. In the standard textbook foundations, the density matrix is therefore not fundamental. However, the axioms for QM become much nicer if one takes the density matrix as fundamental; see the topic ''Postulates for the formal core of quantum mechanics'' in Chapter A1: Fundamental concepts in quantum mechanics of my theoretical physics FAQ at http://www.mat.univie.ac.at/~neum/physfaq/physics-faq.html

Moreover, when analysing actual experiments on the foundations of quantum mechanics, one must represent all states as mixed states to get correct models. These models are of the form of a Lindblad equation for the density matrix of the system.

The usual wave function formalism appears just to be an idealization in which one ignores the effect of the environment (rigorously allowed only at the temperature 0K) and thus arrives at marginally simpler equations.

-
Although the Lindblad equation seems to be preferred by the mathematical physics community it violates either translation invariance symmetry or equipartition and other alternatives such as the Redfield equation, Caldeira-Legget equation... are used by physicists and chemists. Moreover the Lindblad equation is only an approximation to more general equations of motion for the density matrix. –  juanrga Oct 30 '12 at 18:03
@juanrga: What counts in the foundations is quantum optics, where the corresponding experiments are performed. Most models for analyzing quantum optics experiments are based on Lindblad equations. Lindblad equations are the Markov approximation of more complex integro differential equations, which are little used in quantum optical practice except in special circumstances. In any case, working with pure states is inadequate. –  Arnold Neumaier Oct 31 '12 at 12:52
@juanrga: The experimental setup already destroys translation invariance, so that there is no need for this to be preserved in the analysis of a quantum optics experiment. –  Arnold Neumaier Oct 31 '12 at 16:24
Precisely Lindblad modified his own equation (adding a counterterm) that restores translational invariance, but at the cost of violating equipartition. The lost of translational invariance is not an issue in some quantum optics applications, but it is in other experiments where the Lindblad equation gives the wrong answers. Alternative equations, as those cited in my above comment, are then used. –  juanrga Nov 1 '12 at 12:29
We are, possibly, in a minority, but we're not dead yet: in theory, every system is always in a pure state, according to «purists» like me and Dirac. Okay, Dirac is dead and I'm no Dirac.... so there are stil two points of view on this and you have explained one of them pretty well. –  joseph f. johnson Feb 12 '13 at 15:48

Let me try to convince you that the density operator is a mathematical convenience and not a fundamental aspect of quantum mechanics by describing a very general setup for states and observables in both classical and quantum mechanics. This may not directly answer your question, but hopefully it will settle whatever motivated this question.

Briefly,

• Observables $A$ are a Poisson *-algebra. This means a complex algebra $A$ equipped with an antilinear involution $a \mapsto a^{\ast}$ such that $(ab)^{\ast} = b^{\ast} a^{\ast}$ as well as a Lie bracket $\{ -, - \} : A \otimes A \to A$ which is also a derivation with respect to multiplication (and maybe some compatibility between these two structures). Classical examples occur when $A$ is the algebra of complex-valued smooth functions on a symplectic manifold $M$, the involution is pointwise conjugation, and $\{ -, - \}$ is the usual Poisson bracket, and quantum examples occur when $A$ is the algebra of linear operators on a Hilbert space $H$, the involution is adjoint, and $\{ -, - \}$ is the commutator.
• States are *-linear functionals $\mathbb{E} : A \to \mathbb{C}$ on $A$ such that $\mathbb{E}(1) = 1$ and such that $\mathbb{E}(a^{\ast} a) \ge 0$ for all $a$. Classical examples occur when $\mathbb{E}$ is integration against a probability measure on a symplectic manifold $M$. Pure quantum examples occur when $\mathbb{E}(a) = \langle \psi, a \psi \rangle$ for some unit vector $\psi$ in some Hilbert $^{\ast}$-representation of $A$ and you can get more examples by taking linear combinations or more generally integrals of these.

I wrote down what amounts to a justification of this formalism in a somewhat long series of blog posts:

http://qchu.wordpress.com/2011/07/16/the-heisenberg-picture-of-quantum-mechanics/

http://qchu.wordpress.com/2011/08/14/poisson-algebras-and-the-classical-limit/

http://qchu.wordpress.com/2012/08/18/noncommutative-probability/

http://qchu.wordpress.com/2012/09/09/finite-noncommutative-probability-the-born-rule-and-wave-function-collapse/

The Poisson structure is only necessary to understand time evolution; if you just want to understand states, you can safely ignore the first two posts.

Density operators come into the picture as follows. $A$ is sometimes equipped with a canonical linear functional (which may not be everywhere defined); a normalized version of the corresponding state (when it exists) should be thought of as the "uniform distribution." Classical examples occur when this linear functional is given by integration against Liouville measure on a symplectic manifold $M$, and quantum examples occur when this linear functional is given by trace. If $\text{tr}$ denotes this functional, then you can use it to write down a distinguished class of states of the form

$$\mathbb{E}(a) = \text{tr}(\rho a)$$

for some $\rho \in A$. This is the density operator of the state. If $\text{tr}$ satisfies a suitable nondegeneracy condition, $\rho$ can be uniquely recovered from the state it defines. The axioms defining a state require $\rho$ to be self-adjoint and have trace $1$, and it must also have suitable positivity properties (for example in the case of a finite-dimensional matrix algebra it must have non-negative eigenvalues).

You are absolutely correct that every state can be described using a vector in a suitably large Hilbert space (here I am using "Hilbert space" in the physicist's sense, which I understand to really be "inner product space"). This is a corollary of a version of the Gelfand-Naimark-Segal construction which is explained in the third post above.

However, it is worth mentioning that there is an intrinsic definition of pure state in the theory of operator algebras: namely, the space of states has a natural convex structure and you can define a pure state to be an extreme point of the space of states.

So, summary: what is fundamental is a linear functional on the algebra of observables, and everything else comes down to finding convenient ways to write down and analyze such linear functionals.

-
nice answer, but I'd draw the opposite conclusion as I understood Jonathan's question as a question about mixed states, and not their particular representation as density operators; from the model-centric point of view you outline here, mixed states are as natural as pure ones, it's just that the latter ones are extremal; a more physical argument is that the nature of QM makes it impossible to discern between intrinsic (quantum) probabilities of pure states and extrinsic (statistical) probabilities of ensembles as these mix when changing the decomposition of mixed into pure states –  Christoph Oct 30 '12 at 8:16
That is all correct and well written but misses the point! –  MBN Oct 31 '12 at 11:13
@MBN: would you care to enlighten me about what the point is, then? I'm not a physicist, so I wouldn't be surprised if I missed something obvious to physicists. –  Qiaochu Yuan Oct 31 '12 at 17:04
It is contradictory to first say «the density operator is a mathematical convenience and not a fundamental aspect of quantum mechanics» but then sum up by saying «what is fundamental is a linear functional on the algebra of observables». You yourself point out that not every linear functional on the algebra of observables comes from a pure state. –  joseph f. johnson Feb 12 '13 at 15:39
This contradiction can be seen to be «the main contradiction» (remember, always grasp the main contradiction) if you look at the physics: from the first pint of view, what really exists is the system in one pure state or another, but if we do not have enough info, we do some stat mech calculations for a mixed state, etc. But from the last point of view, what is fundamental is knowing all expectation values (or probabilities) for the results of measurements i.e. being able to associate a number to each observable, so a mixed state just as valid as a pure state since it does this equally well. –  joseph f. johnson Feb 12 '13 at 15:45

My impression from the literature is that physicists are still divided on this question. The Quantum Information Theory camp says the latter, but the Quantum Optics people say the former.

A related, but distinct, issue is whether one regards the concept of «open system» as a mere mathematical convenience, or as a fundamental concept. This issue has received less attention. It is striking that in Dirac's basic textbook, «system» means «closed system», just as it always did in Hamiltonian mechanics: there is no other kind of system. Obviously in order to do some practical calculations in physical situations where dissipation or decoherence plays a role, the device of an open system is convenient. The issue of whether it is fundamental is still not settled.

Both issues are related to one's sympathies or lack thereof with «decoherence» as the answer to the puzzle of quantum measurement. In spite of some claims to the contrary, decoherence is not yet accepted by consensus. See Stephen Adler's well argued paper pointing out the serious problems with decoherence when taken as the solution to the quantum measurement problem (everyone agrees that decoherence is sometimes a convenient mathematical device)

Why Decoherence has not Solved the Measurement Problem: A Response to P. W. Anderson. Stephen L. Adler. Stud. Hist. Philos. Sci. Part B 34 no. 1, pp. 135-142 (2003). doi:10.1016/S1355-2198(02)00086-2, arXiv:quant-ph/0112095.

Thermodynamic Limits, Non-commutative Probability, and Quantum Entanglement. Joseph F. Johnson. In Proceedings of the 3rd International Symposium, Quantum Theory and Symmetries (Cincinnati, USA, 10-14 September 2003). arXiv:quant-ph/0507017

and the excellent paper by the great Roger Balian and two assistants,

The quantum measurement process in an exactly solvable model. Armen E. Allahverdyan, Roger Balian and Theo M. Nieuwenhuizen. AIP Conf. Proc. 750, pp. 26-34 (Foundations of probability and physics - 3, Vaxjo, Sweden, 7-12 June 2004). doi:10.1063/1.1874554 arXiv:cond-mat/0408316

all of which should be studied in order to make up your mind on the subject. Balian, as all statmech people, relies heavily on the use of the density matrix without enquiring too closely as to its foundational status. But since there is real important physics in their paper, it (and its precursor zipped files posted at http://www.chicuadro.es/ , H. Green, Nuovo Cimento 9 (1958), 880, written under Schroedinger's influence while at Dublin---but note that decades later, having long escaped Schroedinger's influence, Green adopted an opposing viewpoint) is of key importance to answering your question.

-

Your question is ambiguous. If you are asking if the density operator $\hat{\rho}$ formalism is more fundamental than the state vector $|\Psi\rangle$ formalism, the response is yes, because the density operator formalism applies to open quantum systems for the which no state vector exists. Moreover, it has been shown in last decades that there exists a kind of isolated quantum systems (LPSs) for the which the equivalence between density operators and state vectors is broken and a new kind of solutions (outside the Hilbert space) appear which only can be described in terms of density operators. Some recent Solvay conferences were devoted to such systems.

If you are asking if the density operator formalism is the only way to study such systems (LPS, open...), the response is no, because such systems can be studied using alternatives to the density operator formalism: e.g. in the Wigner-Moyal formulation an open quantum system is described by a Wigner distribution $\rho_W$. However, notice that in both cases the Wigner-Moyal formalism can describe systems beyond the scope of the state vector formalism.

-
Perhaps I could rephrase my question as "Does there exist a quantum system that exists in reality which cannot be mathematically described by a pure state?". According to your answer, it seems that the answer is "Yes" and that an example is given by "open quantum systems". What exactly do you mean by this and how are they not described by pure states? –  Jonathan Gleason Oct 30 '12 at 21:57
You wouldn't have any references for those LPSs, would you? –  leftaroundabout Oct 31 '12 at 13:09
@leftaroundabout: A basic review is The Liouville Space Extension of Quantum Mechanics 1997: Adv. Chem. Phys. 99, 1---120. Petrosky, T.; Prigogine, I. You may also check Phys. Rev. A 56, 3507–3528 (1997). –  juanrga Nov 1 '12 at 12:02
@JonathanGleason: Feynman put it in succinct form in section 2.1 of his lectures on Statistical Mechanics: "When we solve a quantum-mechanical problem, what we really do is divide the universe into two parts--the systems in which we are interested and the rest of the universe. We then usually act as if the system in which we are interested comprised the entire universe. To motivate the use of density matrices, let us see what happen when we include the part of the universe outside the system." –  juanrga Nov 1 '12 at 12:11
@JonathanGleason: When we consider open quantum systems interacting with the surrounds, we find that no state vector reproduces the observables $\langle A \rangle \neq \langle \Psi | \hat{A} | \Psi \rangle$ neither the equation of motion. –  juanrga Nov 1 '12 at 12:13

This is an interesting as still open question.

@Quiaochu Yan The point is that we are using a mathematical model to understand how the world is working. We are using vectors and matrices to make prevision of some feature of the nature. You cannot answer to the question "is the mathematical model that we are using describing completely the nature or not?" staying in the mathematical model! In this case experimental observations are needed.

What physicist are trying to do is to extend quantum mechanics, removing for instance one axioms, and to look if it is still a good mathematical theory, that is if it still in same way make right prevision. This is analog to how we have proven that classical mechanics is not complete. These trials are in the same trend to understand if a density matrix is a fundamental aspect of quantum mechanics or not.

-

I'll throw in two "classical" citations from Landau-Lifshitz, book 5, chapter 5:

The averaging by means of the statisitcal matrix ... has a twofold nature. It comprises both the averaging due to the probalistic nature of the quantum description (even when as complete as possible) and the statistical averaging necessiated by the incompleteness of our information concerning the object considered.... It must be borne in mind, however, that these constituents cannot be separated; the whole averaging procedure is carried out as a single operation, and cannot be represented as the result of succesive averagings, one purely quantum-mechanical and the other purely statistical.

And then this:

It must be emphasised that the averaging over various $\psi$ states, which we have used in order to illustrate the transition from a complete to an incomplete quantum-mechanical description has only a very formal significance. In particular, it would be quite incorrect to suppose that the description by means of the density matrix signifies that the subsystem can be in various $\psi$ states with various probabilities and that the average is over these probabilities. Such a treatment would be in conflict with the basic principles of quantum mechanics.

Taking these into account, I'd say that density matrix is actually more "fundamental" than just pure states description.

-
The way most people interpret Landau, and the way I read these two quotations, is the exact opposite of the way you read them. This, I think, shows that although he was a great physicist, he was not good at foundations. J.S. Bell has a very useful critique of Landau's discussion of measurement. Look at these quotes again: Landau says the density matrix is because of the incompleteness of our information concerning the object considered, i.e., it is not fundamental, it is subjective. –  joseph f. johnson Feb 11 '13 at 15:39
Bell's careful critique of Landau, Gottfried, and van Kampen (as foundationalists) was later published in Physics World 3 (1990), 33 and is freely available at chicuadro.es/BellAgainstMeasurement.pdf –  joseph f. johnson Feb 11 '13 at 19:49