# Why can't every quantum state be expressed as a density matrix/operator?

It was my previous impression that all quantum states in a Hilbert space can be represented using density matrices and that's already the most general formulation of a quantum state. Then I came across yuggib's comment here:

Everything would be so easy if there was the one-to-one correspondence you are describing. Sadly, there are many very strong suggestions that this should not be the case. The existence of uncountably many inequivalent irreducible representations of the canonical commutation relations for quantum fields is one of such suggestions. Another is the fact that not every quantum state can be represented, in a given (irreducible) representation, as a ray in Hilbert space (or as a density matrix, actually).

It seems even density matrices don't provide a good enough definition for the "state" of a quantum system, although I don't quite understand why. According to Schuller, in the general formulation of quantum mechanics, the state of a quantum system is defined as a positive trace-class linear map $$\rho: \mathcal{H} \to \mathcal{H}$$ for which $$\mathrm{Tr}(\rho)=1$$. How exactly does this definition encapsulate what density matrices cannot? Or are these two actually equivalent and I'm missing some point here?

I'm further confused because Wikipedia clearly states: "Describing a quantum state by its density matrix is a fully general alternative formalism to describing a quantum state by its ket (state vector) or by its statistical ensemble of kets." and that directly contradicts yuggib's comment.

†: Or rather, density operators, if dealing with infinite dimensional Hilbert spaces.

• In introductory material, yeah, all states are just density matrices where $\rho^2 = 1$. But in quantum field theory, where the Hilbert space is very very infinitely dimensional, things become a bit murkier. For instance, "Haag's theorem" shows that an interacting theory of QFT doesn't exist with all the properties we assume it has. (Having said that, this theorem is probably a bit unfair.) Anyway, I think density matrices are a perfectly fine way to talk about states, but probably there are subtle issues in QFT that most people don't care about because they're not too important. – user1379857 May 29 at 1:11
• Everything is murkier in infinite dimensional spaces. Of course, you can formulate all the theories we know and love in finite dimensional spaces, rendering all these concerns about rigor immaterial, but for some reason this possibility is not regarded as interesting. – knzhou May 29 at 1:32
• Interacting QFTs have resisted rigorous mathematical definitions. See, for instance, the "Yang–Mills existence and mass gap" Clay Institute prize. Yang–Mills theories are some of the most important in physics, and yet we don't even know if they "exist" rigorously. (However, maybe this isn't a problem if you assume a more fundamental theory of quantum gravity kicks in at short distances and changes everything.) – user1379857 May 29 at 1:57
• It is strongly false that everyrhing in QM can be formulated in a finite dim Hilbert space, considering the infinite dimensional case as a "straightforward" limit. E.g., there are no X and P satisfying CCRs in finite dim space, so no limit can be considered. In finite dim, independent systems are always described in terms of tensor product. In infinite dim, this fact is untenable in relevant situations in qft in particular. – Valter Moretti May 29 at 4:36
• @Blue For bounded operators (and trace-class includes boundedness) positivity implies selfadjointness, so it is not necessary to mention it. Positive and positive semidefinite are the same notion in this context... – Valter Moretti May 29 at 9:02

The statement by yuggib is correct. To put it in perspective, I'll start with a completely general formulation, and then I'll show how vector-states and density operators fit into that picture. I won't try to be mathematically rigorous here, but I'll try to give an overview with enough keywords and references to enable further study.

## State = normalized positive linear functional

Every quantum state, pure or mixed, can be represented by a normalized positive linear functional on the operator algebra. Such a functional takes any operator $$X$$ as input and returns a single complex number $$\rho(X)$$ as output, with nice properties like $$\begin{gather*} \rho(X+Y)=\rho(X)+\rho(Y) \hskip2cm \rho(cX)=c\rho(X) \\ \rho(X^*X)\geq 0 \hskip2cm \rho(1)=1 \end{gather*}$$ for all operators $$X,Y$$ and complex numbers $$c$$. I'm using an asterisk both for complex conjugation and for the operator adjoint, and I'm writing $$1$$ both for the identity operator and for the unit number. I'm also considering only bounded operators to keep the statements simple. This is always sufficient in principle, even though we normally use some unbounded operators in practice because it's convenient.

"Normalized positive linear functional" is a long name for a very simple thing. It also has a shorter name: mathematicians often just call it a state (see Wikipedia), and I'll use that name here. In [1], it is called an algebraic state to distinguish it from other usages of the word "state."

A state is called mixed if it can be written as $$\rho(X) = \lambda_1\rho_1(X)+\lambda_2\rho_2(X)$$ for all $$X\in{\cal A}$$, where $$\rho_n$$ are two distinct states and where the coefficients $$\lambda_n$$ are both positive real numbers (not zero). A state that cannot be written this way is called pure.

This is all completely general. It works just fine in everything from a single-qubit system to quantum field theory. In contrast, using a density operator to represent a state is mathematically less general. The following paragraphs address how vector-states and density matrices fit into the more general picture described above.

## Vector states and density matrices / density operators

The GNS theorem says that a state can always be implemented as $$\rho(\cdots) =\frac{\langle\psi|\cdots|\psi\rangle}{ \langle\psi|\psi\rangle}$$ where $$|\psi\rangle$$ is a single vector in some Hilbert-space representation of the operator algebra. Even mixed states can always be implemented this way. The catch is that the required Hilbert-space representation is not necessarily irreducible, and we may need to switch to different Hilbert-space representations to implement different states this way. The Hilbert-space representation of the operator algebra is irreducible if and only if the state is pure [2][3].

A state $$\rho$$ is called a normal state if an operator $$\hat\rho$$ (a density matrix or density operator) exists such that [4] $$\rho(\cdots) = \text{Trace}(\cdots \hat \rho).$$ The fact that this kind of state has a special name suggests that it is a special kind of state — that not every state can be expressed this way. This is confirmed in [5], where counterexamples are described by Valter Moretti. The Math SE question [6] also asks for a counterexample, and it has an answer.

## Conclusion

This is all consistent with yuggib's statement

not every quantum state can be represented, in a given (irreducible) representation, as a ray in Hilbert space (or as a density matrix, actually).

The statement needs to be parsed carefully, though: the words given and irreducible are important. The Wikipedia page that said "Describing a quantum state by its density matrix is a fully general alternative..." might be referring to a less-general context, like finite-dimensional Hilbert spaces, or might be implicitly using a less-general definition of "state." That doesn't mean the Wikipedia page is wrong; it just means that — as always — we need to beware of equivocation.

References:

[1] Valter Moretti (2013), Spectral Theory and Quantum Mechanics (A 2018 edition is also available; I cited the 2013 version because it's the one I had on hand when writing this answer)

[2] Proposition 1.8 in https://arxiv.org/abs/math-ph/0602036

[3] Theorem 14.12 in [1]

[6] "Non normal state" (https://math.stackexchange.com/q/2962163)

Addendum: This answer has been downvoted a couple of times. I don't know why (no comments were left), but I'm adding the following clarification in case it addresses the concern:

If the question had been "Are normal states sufficient for all practical purposes?" then the answer would surely be yes. But that wasn't the question. The quesetion asked for the reason behind a specific mathematically-minded statement about states on operator algebras, and that's what this answer tries to address.

• @ValterMoretti I didn't see your comments below the question until after I posted my answer (I'm a slow writer), but my answer cites two sources written by you. Please correct me if I've misrepresented anything in my answer. – Chiral Anomaly May 29 at 5:00
• Thank you very much for quoting my book! Actually there is a 2018 edition including some new material also concerning the discussed issue... I have only a minir remark about your post: the second condition you wrote defining $\rho$ should be replaced by $\rho(cA) = c\rho(A)$. Actually, the condition you wrote is a consequence of linearity and positivity of $\rho$ (positivity us your penultimate requirement). – Valter Moretti May 29 at 5:08
• This is such a clean answer, +1. – gented Jun 3 at 8:43

I think the statement is simply mistaken. The first part of the bolded quote, "[N]ot every quantum state can be represented, in a given (irreducible) representation," is accurate, because only pure states exist as rays in the Hilbert space. In fact, the whole point of the density matrix formulation is that it allows for more general states, which are not pure states. For a pure state, the density matrix is effectively projection operator onto that state (thus satisfying $$\rho^{2}=\rho$$), but the density matrix can also be a probabilistically weighted sum of such projection operators. (What exactly these mixed states mean brings us to the puzzle of the correct interpretation of quantum mechanics; however, from a practical standpoint, they do exist, at least in some sense.)

I suspect that the author of that quote simply overgeneralized. For non-pure states, there is not a representation in terms of a density matrix $$\rho$$ that satisfies $$\rho^{2}=\rho$$. Many pedagogical treatments of the density matrix start by only considering the density matrices for pure states, for which $$\rho^{2}=\rho$$ is a consistency condition; in fact, some treatments never even deal with the more general case. However, I personally think such an approach is foolish, since the most important motivation for the density matrix formulation of quantum mechanics is precisely its capacity to handle mixed states.