Von Neumann entropy: How do we get $S(\rho)=-\sum_i p_i \ln(p_i)$?

I would like to understand why the Von Neumann entropy can be written like this :

$$S(\rho)=-\sum_i p_i \ln(p_i)$$

as written here: https://en.wikipedia.org/wiki/Von_Neumann_entropy

Indeed, if I want to prove it from the definition I would do:

$$S(\rho)=-Tr(\rho \ln(\rho))=-Tr(\sum_{ij} p_i \ln(p_j)|\psi_i \rangle\langle\psi_i|\psi_j\rangle\langle\psi_j|) \\=-\sum_{kij} p_i \ln(p_j) \langle \phi_k|\psi_i\rangle\langle\psi_i|\psi_j\rangle\langle\psi_j|\phi_k\rangle \\ =-\sum_{ij} p_i \ln(p_j) |\langle \psi_i | \psi_j \rangle|^2$$

As we don't have orthogonal states for our density matrix in general, I don't see how we could end up with the formula of the wikipedia page.

• Where did you find that first expression for $S$? I don't see it in the link as you write it there. – secavara Jan 28 '18 at 14:54
• @secavara I edited sorry – StarBucK Jan 28 '18 at 14:57
• Ok. They are using the spectral decomposition theorem (link here). Applying this to the density operator, which is hermitian, we can always find an orthonormal basis of its eigenvectors for our Hilbert space: states with different eigenvalues are guaranteed to be orthogonal and even in degenerate subspaces we can build an orthonormal basis of eigenvectors. The density matrix is then build as the linear superposition of the corresponding projectors multiplied by their eigenvalues, i.e. $\rho = \sum_j p_j |\psi_j \rangle \langle \psi_j|$. – secavara Jan 28 '18 at 15:09

The density operator is hermitian. This means that you can find one orthonormal basis $|\phi_i\rangle$ of eigenvectors for it. By definition this means that there are real numbers $p_i$ such that

$$\rho |\phi_i\rangle = p_i |\phi_i\rangle.$$

Now, it is known that if $f(x)$ is one ordinary function, and $A$ is one hermitian operator with orthonormal basis of eigenvectors $|a_i\rangle$ then we define $f(A)$ on this basis to be

$$f(A)|a_i\rangle=f(a_i)|a_i\rangle,$$

which in turn defines $f$ on the whole Hilbert space, since the $|a_i\rangle$ are a basis.

Now this is how $f(\rho)=\rho \ln \rho$ is defined. In the basis of $\rho$ we have

$$f(\rho)|\phi_i\rangle=p_i \ln p_i |\phi_i\rangle.$$

Now remember you can compute the trace in any basis you want. We compute it in this basis, remembering that $f$ has matrix elements:

$$\langle \phi_j |f(\rho)|\phi_i\rangle = p_i \ln p_i \delta_{ij}.$$

Thus the trace is exactly

$$S(\rho)=-\operatorname{Tr}(\rho\ln \rho)=-\sum_{i}p_i \ln p_i.$$

Now, as a remark, in my opinion things can be thought the other way around. This latter expression for entropy was known before QM, with $p_i$ being the probabilities for the microstates. In order to generalize to Quantum Mechanics, remember that $\rho$ represents one ensemble, so that when you have a mixed state, you don't actualy know the actual microstate. On the other hand $\rho$ encodes the probabilities for the microstates as those $p_i$ above.

Thus one could start with the previous knowledge of what $S$ should be in terms of these $p_i$ and arrive at a general expression involving just $\rho$. In simple terms: $S(\rho)$ is defined to yield this result.