I am trying to understand how quantum states are a generalization of probability distributions and have some issues understanding purifications. A mixed quantum state $\rho_A$ can always be purified into a pure state $\vert\psi_{AB}\rangle$ such that $\text{Tr}_B(\vert\psi\rangle\langle\psi\vert) = \rho_A$.

What is the classical analogue of this for probability distributions? If I have some probability vector $P_A$, is there any meaningful notion of a pure state (i.e. a probability vector of the form $(1,0,0...0)$ on a larger space) whose marginal distribution turns out to be $P_A$?

Of course, every state $\rho_A$ can be diagonalized and then it is effectively classical. One can then purify this diagonal state but I think what happens here is that one gets the purification in the Schmidt basis. This is not of the form

$$\vert\psi\rangle\langle\psi\vert \neq \begin{pmatrix} 1\\ &\!\! 0\\ &&0\\ &&& \ddots \end{pmatrix}$$

and so we cannot go from this purification to a probability vector.

So what is the correct way to think about purifications for "classical" states i.e. probability vectors? Any general comments or links to resources that discuss these are greatly appreciated!


2 Answers 2


The answer is no -- tracing out part of a system always reduces the entropy, so the marginal distribution will always be less mixed.

This is different for quantum state -- a part of a system can have more randomness than the system as a whole.


Just to add a couple more equations to the other answer, consider a (classical, discrete) joint probability distribution $(p_{ij})_{ij}$. Let the corresponding marginal distribution be $(q_i)_i$ with $q_i=\sum_j p_{ij}$. Then $$S(q)=-\sum_i q_i \log q_i=-\sum_{ij}p_{ij}\log\left(\sum_k p_{ik}\right) \le -\sum_{ij}p_{ij}\log p_{ij}=S(p).$$

On the other hand, in the quantum case, consider the maximally entangled two-qubit state $$\newcommand{\ket}[1]{\lvert #1\rangle}\ket\Psi\equiv\frac{1}{\sqrt2}(\ket{0,0}+\ket{1,1}).$$ This, when measured in the computational basis, corresponds to the probability distribution $p_{ij}=\frac12(\delta_{i,0}\delta_{j,0}+\delta_{i,1}\delta_{j,1})$, which has entropy $S(p)= \log2 .$ However, and this is a crucial point, in the quantum case the entropy depends on the measurement basis, and if a state is pure we can always find a measurement basis with respect to which the entropy is zero.

The corresponding marginal state is totally mixed state: $\operatorname{Tr}_B(\ket\Psi\langle\Psi\rvert)=I/2$. This is not a pure state anymore, and in any measurement basis it corresponds to the outcome probabilities $q_1=q_2 = 1/2$, and thus to the entropy $S(q)=\log2$.

Therefore in the quantum case the marginal distribution can have larger entropy than the joint distribution, which as we showed above is not possible for classical distributions.

  • $\begingroup$ Thank you! It's an interesting point that the quantum entropy can be seen as computing the classical entropy on the resulting probability distribution after a measurement. So if I understand correctly, one has to minimize this over all possible measurements and this will reveal that the right measurement is the one in the eigenbasis of the state. $\endgroup$ Commented Aug 10, 2020 at 15:50
  • 1
    $\begingroup$ @user1936752 yes indeed. That turns out to be equivalent to computing the entropy of the eigenvalues of the density matrix (which are always a probability distribution, due to states being normalised) $\endgroup$
    – glS
    Commented Aug 10, 2020 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.