# Understanding conditional Entropy between two physical states

From the paper: https://arxiv.org/abs/1303.4686

Let us consider an ensemble of N identical quantum systems with Hamiltonian H and density matrix $$\rho^{(N)}$$ which is diagonal in the energy eigenbasis and has $$\rho^{(N)} = \mathrm{diag}\{P_1,P_2,P_3,...\}$$ values which sum to 1.

Now let us consider the corresponding thermal state $$\tau^{(N)}(\beta)$$ with the same Von-Neumann entropy such that

$$S(\rho^{(N)}) = S(\tau^{(N)}(\beta))$$

I can calculate the energy difference between both states

$$\Delta E = \mathrm{Tr}(\rho^{(N)} H)- \mathrm{Tr}(\tau^{(N)}(\beta) H) = \mathrm{Tr}[(\rho^{(N)}-\tau^{(N)}(\beta)) H]\tag{1}$$

But now the paper I'm reading claims that this energy difference from eq. (1) can be expressed as

$$\Delta E = \mathrm{Tr}[(\rho^{(N)}-\tau^{(N)}(\beta)) H] = T\cdot S(\rho^{(N)}\mid \mid\tau^{(N)}(\beta))$$

where $$S(\rho^{(N)}\mid \mid\tau^{(N)})$$ is the conditional entropy. (Equation (11) in the paper)

The only conditional entropy I've ever known was related to measurements and outcomes.

https://en.wikipedia.org/wiki/Conditional_entropy

Can someone maybe just conceptually explain to me what $$S(\rho^{(N)}\mid \mid\tau^{(N)})$$ stands for? Especially because both states should have the same entropy, I am confused.

• arxiv.org/pdf/1303.4686.pdf Equation (11). – CatoMaths Jan 13 at 14:32

It is not the conditional entropy, it is the relative entropy -- the former would be denoted by $$S(\rho|\tau)$$ (i.e. only one vertical bar). It seems that the authors used the wrong word in their paper.

The relative entropy is defined as $$S(\rho\|\tau)=\mathrm{tr}[\rho(\log\rho-\log\tau)]$$

Now let us prove the above relation. (I just write $$\rho$$ and $$\tau$$, with $$\tau=e^{-\beta H}/Z$$, and $$S(\rho)=S(\tau)$$.)

We have \begin{align*} S(\rho\|\tau) & = \mathrm{tr}[\rho\log\rho]-\mathrm{tr}[\rho\log\tau]\\ & = \mathrm{tr}[\tau\log\tau]-\mathrm{tr}[\rho\log\tau]\\ & = \mathrm{tr}[(\tau-\rho)\log\tau]\\ & = \mathrm{tr}[(\tau-\rho)(-\beta H-\log Z)]\\ & = \beta\mathrm{tr}[(\rho-\tau)H]\ , \end{align*} which is the claimed relation. Here, in the second line I have used that $$\mathrm{tr}[\rho\log\rho]=S(\rho)=S(\tau)=\mathrm{tr}[\tau\log\tau]$$, then the definition of $$\tau$$ as a thermal state, and finally, that $$\mathrm{tr}[\rho-\tau]=0$$, so that the $$\log Z$$ term drops out.

• arxiv.org/pdf/1303.4686.pdf Equation (11). I'm afraid it is the conditional entropy. Even though maybe the author used the wrong notation. – CatoMaths Jan 13 at 14:32
• How do you know the authors didn't mean to say "relative entropy"? Why do you think their mistake is in the notation, not in the words? – Norbert Schuch Jan 13 at 14:38
• @CatoMaths Found the proof. I think it is always crucial to provide a link to the paper, for many many reasons. – Norbert Schuch Jan 13 at 14:48
• Perfect mathematically everything makes sense now! Thank you! But I stil struggle with the conceptual understanding. The relative entropy is used to quantify distinguishability between two states. It can be used as a measure of entanglement. So by that logic if $\Delta E$ is small the two states $\rho$ and $\tau$ are not 'operationally distinguished' and thus entangled? Somehow this statement relates entanglement to the energy difference between the two states. That's fair. But what does distinguishability mean in this context? – CatoMaths Jan 13 at 16:02
• Maybe then this deserves an upvote ;) The other is a different question, I'd say. I wouldn't want to clog either the question or the answer, I don't see how anyone would profit from that. But I wondered myself after answering, it is not completely obvious. But one would have to think about the notion of distinguishablity which the rel. entropy measures, which is in an asymptotic sense, and the paper indeed makes statements about those two state in the aymptotic case. – Norbert Schuch Jan 13 at 16:06