I have found the Gibbs entropy of a simple system and the summed Gibbs entropy of its subsystems. I believed that these values should be equal. They are not. I'd be grateful for any ideas regarding where I have went wrong.

Consider an isolated system, $S$, which can be split into two subsystems: $L$ and $R$.

There are ten microstates of system $S$. Subsystem $L$ can be in one of three microstates, $A_1^L$, $B^L_1$ and $B^L_2$. Subsystem $R$ can be in one of seven microstates, $A_1^R$, $A_2^R$, $A_3^R$, $A_4^R$, $B^R_1$, $B^R_2$ and $B^R_3$.

If subystem $L$ is in a microstate of type $A$ then so is $R$ (same for $B$ microstates). This is due to conservation of energy. Let $E(A^L)$ be the energy of the subsystem $L$ in an $A$ state. Likewise for other terms of this form. Let $E(A^L)+E(A^R)=E(B^L)+E(B^R)=U$, where $U$ is the total energy of $S$. Let $E(A^L) \neq E(B^L)$.

The ten possible microstates of the overall system $S$ are thus:

$A_1^L$ $A_1^R$,

$A_1^L$ $A_2^R$,

$A_1^L$ $A_3^R$,

$A_1^L$ $A_4^R$,

$B_1^L$ $B_1^R$,

$B_1^L$ $B_2^R$,

$B_1^L$ $B_3^R$,

$B_2^L$ $B_1^R$,

$B_2^L$ $B_2^R$,

$B_2^L$ $B_3^R$

Each microstate of $S$ is equally likely. Thus the probability $S$ is in any one of these ten microstates is $\frac{1}{10}$.

The Gibbs total entropy of system $S$, given by $-\sum_i P_i \log P_i$, is simply $\log 10$. (Let $k_B=1$)

Let us find the Gibbs entropy of subsystem $L$. The probability it is in state $A_1^L$ is $\frac{4}{10}$. This is because there are four overall system microstates where $L$ is in this state. Likewise $L$ is in state $B_1^L$ with probability $\frac{3}{10}$, equivalently for state $B_2^L$. Thus the entropy of subsystem $L$ is


Likewise the entropy of subsystem $R$ is

$-[\frac{1}{10}\log\frac{1}{10}+\frac{1}{10}\log\frac{1}{10}+\frac{1}{10}\log\frac{1}{10}+\frac{1}{10}\log\frac{1}{10} + \frac{2}{10}\log\frac{2}{10}+\frac{2}{10}\log\frac{2}{10}+\frac{2}{10}\log\frac{2}{10}]$


The sum of subsystem $R$ and subsystem $L$ entropies is,

$2\log10 -\frac{4}{10}\log4 -\frac{6}{10}\log 6$

$=2\log10 -\frac{1}{10}\log 256-\frac{1}{10}\log 46656$

$=2\log10 -\log5.1$

$\neq \log 10$

So, the total entropy is not equal to the summed entropy of the system's subsystems. What has went wrong? Why can Gibbs entropy not be manipulated in this way?

  • $\begingroup$ If $L$ has 1 $A$-type and 2 $B$-types, $R$ has 4 $A$-types and 1 $B$-type, you have 6 states, not 10. So $R$ had 3 $B$-type states. Suggested reading: en.wikipedia.org/wiki/Conditional_entropy $\endgroup$ Commented Nov 11, 2019 at 15:23
  • 1
    $\begingroup$ Sorry, that was a typo. R has 3 B-type states. Typo now corrected. $\endgroup$ Commented Nov 11, 2019 at 15:33

1 Answer 1


Long story short: the entropy of two subsystems only decomposes into a sum when the systems are approximately independent (in a statistical sense). If they are strongly coupled, as the ones in the example are, you need to turn to conditional entropy to manage the sum.

Long story short, the correct equation is: \begin{align} H(X,Y) & = H(X|Y) + H(Y) \end{align} because the definition of how joint probability $P(X,Y)$ and conditional probability $P(X|Y)$ is \begin{align} P(X,Y) & = P(X|Y) P(Y). \end{align} When $X$ and $Y$ are independent, $P(X|Y) = P(X)$.

For completeness, conditional entropy is defined as \begin{align} H(X|Y) & = - \sum_{X,Y} P(X,Y) \ln P(X|Y) \end{align} and, if $P(X|Y) = P(X)$ then you can show that $H(X|Y) = H(X)$.

I leave applying this to the particular example in the question as an exercise for the reader.

  • 1
    $\begingroup$ It may be worth adding that "when the subsystems are (strongly) coupled" the total(joint) entropy is always less than the sum of the subsystems. We see it in OP example $\endgroup$ Commented Nov 12, 2019 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.