# What is the entropy of a mixed state in classical physics?

Consider a classical system which admits certain macroscopic level of description. It is known, that for two pure states $\omega_1$ and $\omega_2$ on this level of description the entropy of the system is $S_1$ and $S_2$ accordingly.

Next, let us consider a mixed state: $$\omega_\text{m} = \alpha \omega_1 + (1-\alpha) \omega_2$$

What is the entropy of $\omega_\text{m}$ in terms of $S_1$ and $S_2$?

I think there should be nothing special regarding the sum of greater number of pure states, so I chose the simplest case of two pure states. If my assumption is not true, please indicate.

For the used notion of states see an entry in n-Lab on observables and states.

• Should the entropy not be zero for pure states? (And the entropy of the mixture of states simply be $S = - \alpha \ln \alpha - (1 - \alpha) \ln (1 - \alpha)$ by definition?). – Sebastian Riese Jul 13 '15 at 11:53
• @SebastianRiese State being pure has nothing to do with zero or nonzero entropy. It has to do whether it gives a unique value for all the observables (pure state) or probability distributions for their values (mixed state). For example a pure state for a macroscopical description of a common fluid is given by the fields of density, velocity and temperature. Obviously such description implies the fluid has a nonzero entropy. A mixed state might be of interest in turbulence research when you don't know the fluid velocity at a point, but rather assume there is a probability distribution for it. – Yrogirg Jul 13 '15 at 12:34
• So it is not a pure microstate, but rather a "pure macrostate". Did not meet that nomenclature before, my mistake. – Sebastian Riese Jul 13 '15 at 12:40
• @Yrogirg so you have a system that can now exhibit two micro-states $\omega_1$ and $\omega_2$ with different weights $\alpha$ and $1-\alpha$. So that for certain time of observation $T$ it exhibits $\omega_1$ a fraction $\alpha T$ of the time and exhibits $\omega_2$ a fraction $(1 - \alpha)T$ of the time. Is this coherent with your question? – rmhleo Jul 15 '15 at 9:38
• @rmhleo I can't admit that this would be the only interpretation, but I'd say that yes, one can take it, provided the states do not change in time. – Yrogirg Jul 15 '15 at 12:16

In macroscopic units it should be $$S=-R\alpha \log(\alpha e^{-S_1/R})-R(1-\alpha)\log\Big(1-\alpha)e^{-S_2/R}\Big) \\=\alpha \Big(S_1-R\log\alpha\Big)+(1-\alpha)\Big(S_2-R\log(1-\alpha)\Big),$$ where $R$ is the universal gas constant. In the pure case, this reduces to the textbook formula.
But such a formula cannot be true in general. The general formula is $$S=\langle-R\log\rho\rangle=-R\ Tr (\rho\log\rho),$$ where the trace is over the microstates. Now represent $\rho$ as a mixture of two independent distributions. If system $k=1,2$ is in the classically pure but quantum mixed state $\rho_k$ then the mixed classical state has density matrix $\rho=\alpha\rho_1+(1-\alpha)\rho_2$, If one only knows the entropies of the states $k=1,2$ then $S_k=-Tr\ \rho_k\log\rho_k$ and $$S=-Tr\ \rho\log\rho=-Tr\Big(\alpha\rho_1+(1-\alpha)\rho_2\Big)\log\rho =-\alpha\ Tr\rho_1\log\rho-(1-\alpha)\ Tr\ \rho_2\log\rho,$$ which cannot be simplified further without making approximations or assumptions. The formula given is the most reasonable ''simplification'' independent of other data about the partial states.
• It seems for me that it is actually $\langle -R \log \frac{\rho}{\exp(S/R)} \rangle$. How to get this? This is not obvious at all for me, otherwise I wouldn't be asking the question. Is this derivation available anywhere in the literature? Probably not physics textbooks, but the ones on information theory. – Yrogirg Jul 17 '15 at 13:07