Although it seems simple, I can't get the derivation correct. Here is my reasoning:

$P(S)=P(A)P(B)$

Where P is the probability and S, A, and B denote different systems.

$S_A=-P(A)\ln P(A)$ and $S_B=-P(B)\ln P(B)$

Then $$S_{S}=-P(S)\ln P(S)=-P(S)\ln P(A)P(B)=-P(S)\ln P(A)-P(S)\ln P(B)$$

The problem is, since $P(A)\neq P(S)$ and $P(B)\neq P(S)$, how can entropy be additive?

• I'm pretty confused, because it looks like you start by assuming that $P(S)=P(A)P(B)$ and then show that it contradicts an assumption that $P(S)\ne P(A)P(B)$, which of course shouldn't be surprising. Have I misunderstood what you're trying to show? – Nathaniel Mar 1 '16 at 5:41
• Entropy is only additive in systems without interaction, so your assumption is wrong, already. – CuriousOne Mar 1 '16 at 6:53
• @Nathaniel if I assume P(S)=P(A)P(B) (I don't know if it's a correct statement) then entropy cant be additive because $-P(S)lnP(A)-P(S)lnP(B) \neq -P(A)lnP(A)-P(B)lnP(B)$. That's what I tried to say. – SaudiBombsYemen Mar 1 '16 at 18:27
• @SaudiBombsYemen ah, I see. But when you sum over A and B, they are equal after all. (It that isn't clear, let me know and I'll post an answer.) – Nathaniel Mar 2 '16 at 3:46
• @Nathaniel hmm... I think the problem is that my level of statistics and calculus is still highschool one (last year) so yeah, it'd be perfect if you explained me what you mean :) – SaudiBombsYemen Mar 2 '16 at 15:47

If you consider a system $C$ and two subsystems $A,B$ with associated probability distributions $p_C,p_A,p_B$ and want the entropy to add, you must assume the subsystems are independent in the sense that $p_C(X) = p_A(Y)p_B(Z)$ where $Y$ and $Z$ are a partitioning of the variables $X$ belonging to $C$. Then, the total entropy is \begin{align} S_C & = \langle -\ln(p_C)\rangle\\& = -\int p_C(X)\ln(p_C(X))\mathrm{d}X \\ & = -\int p_A(Y)p_B(Z)(\ln(p_A(Y))+\ln(p_B(Z))\mathrm{d}Y\mathrm{d}Z \\ & = -\int p_A(Y)p_B(Z)\ln(p_A(Y))\mathrm{d}Y\mathrm{d}Z - \int p_A(Y)p_B(Z)\ln(p_B(Z)\mathrm{d}Y\mathrm{d}Z \\ & = S_A+ S_B \end{align} because $\int p_A(Y)\mathrm{d}Y = 1$ and likewise for $B,Z$.