# Von Neumann entropy vs Shannon entropy for a quantum state vector

Is there some kind of information about a quantum system, which we might derive from von Neumann entropy, which is impossible to deduce from Shannon entropy?

Let's say, that we have a bipartite system consisting of two qubits $$A$$ and $$B$$. These qubits might be in an entangled state $$|\Phi^+\rangle = \frac{1}{\sqrt{2}}(|00\rangle + |11\rangle) = \frac{1}{2} (1\ 0\ 0\ 1)^T$$. We can obtain a density matrix representing this system as

$$\rho^+ = |\Phi^+\rangle \langle \Phi^+| = \frac{1}{2}\begin{pmatrix} 1 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 1 \\ \end{pmatrix}.$$

Then, we can get the reduced density matrix of the subsystem $$A$$:

$$\rho_A^+ = Tr_B(\rho^+) = \frac{1}{2}\begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix}.$$

Now, we might want to calculate the von Neumann entropy:

$$S_{vN} = -Tr(\rho_A^+ \log_2 \rho_A^+) = 1.$$

But, if we just treat the squares of coefficients $$c_i$$ of a state $$|\Phi^+\rangle = \sum_i c_i |i\rangle$$ in a $$\{|00\rangle, |01\rangle, |10\rangle, |11\rangle\}$$ basis as probabilities (which is the case for a quantum state), we get exactly the same value of Shannon entropy as von Neumann entropy:

$$S_S = -\sum_i |c_i|^2 \log_2 |c_i|^2 = 1.$$

• Coefficients $c_i$ in which basis?? This will be basis-dependent, the von Neumann entropy of $\rho_A$ won't. If a Schmidt basis: yes, it is the same. If in any basis: Can take any value between 0 and 2*log(d). Dec 12, 2020 at 17:51
• I meant something like $|\Phi^+\rangle = c_0 |00\rangle + c_1 |01\rangle + c_2 |10\rangle + c_3 |11\rangle$. So if these are the same, isn't it just more efficient to compute the entropy using the second formula? If so, why we would need to define entanglement entropy in such a complicated manner using density matrices? Dec 12, 2020 at 19:21
• No, this is not the same - except for the state you tried. Why don't you try a second and a third example? This is always a good idea to try more than one example! Dec 12, 2020 at 19:59
• Do you know about the Schmidt decomposition? If not, you should first read up about it. Dec 14, 2020 at 15:15
• I'm not sure what you're asking. If I remember correctly, an early characterization of the von Neumann entropy of a state was as the minimum outcome uncertainty (in the sense of Shannon's entropy) among all extremal measurements that can be made on that state (excluding the trivial 1-outcome measurement of course). There's an interesting discussion about this by Slater, with further references doi.orgl10.1016/0375-9601(91)90371-E Dec 14, 2020 at 21:09

Thanks to the comment of @Norbert Schuch I finally understand, that Shanon entropy calculated from a state vector represented in the Schmidt basis, is the same as von Neumann entropy.

Let's consider a state $$|\psi\rangle = \frac{1}{4}|00\rangle + \frac{1}{\sqrt{2}}|01\rangle + \frac{\sqrt{3}}{4}|10\rangle + \frac{1}{2}|11\rangle$$. If we treat the squares of the coefficients as probabilities, we get Shanon entropy:

$$S_S = -\frac{1}{16} \log_2\frac{1}{16} -\frac{1}{2} \log_2\frac{1}{2} -\frac{3}{16} \log_2\frac{3}{16} -\frac{1}{4} \log_2\frac{1}{4} \approx 1.18.$$

By creating density matrix from the state vector as $$\rho = |\psi\rangle \langle \psi|$$, and computing entanglement entropy as described in the initial question, we get von Neumann entropy:

$$S_{vN} \approx 0.148.$$

Now, to represent initial state $$|\psi\rangle$$ in the Schmidt basis, we have to do an SVD of this state. We begin by representing the state vector as a matrix:

$$|\psi\rangle = \begin{pmatrix} \frac{1}{4} \\ \frac{1}{\sqrt{2}} \\ \frac{\sqrt{3}}{4} \\ \frac{1}{2} \end{pmatrix} \rightarrow M_\psi = \begin{pmatrix} \frac{1}{4} & \frac{1}{\sqrt{2}} \\ \frac{\sqrt{3}}{4} & \frac{1}{2} \end{pmatrix}.$$

$$SVD$$ gives singular values equal to $$0.98286148$$ and $$0.18434563$$. These are the only non-zero entries in the state vector written down in the Schmidt basis. Now, if we compute the Shanon entropy of squares of these singular values we get

$$S_S' = -|0.98286148|^2 \log_2 |0.98286148|^2 - |0.18434563|^2 \log_2 |0.18434563|^2 \approx 0.148,$$

which is exactly the same as calculated previously von Neumann entropy.