Although it seems simple, I can't get the derivation correct. Here is my reasoning:

$P(S)=P(A)P(B)$

Where P is the probability and S, A, and B denote different systems.

$S_A=-P(A)\ln P(A)$ and $S_B=-P(B)\ln P(B)$

Then $$S_{S}=-P(S)\ln P(S)=-P(S)\ln P(A)P(B)=-P(S)\ln P(A)-P(S)\ln P(B)$$

The problem is, since $P(A)\neq P(S)$ and $P(B)\neq P(S)$, how can entropy be additive?

• I'm pretty confused, because it looks like you start by assuming that $P(S)=P(A)P(B)$ and then show that it contradicts an assumption that $P(S)\ne P(A)P(B)$, which of course shouldn't be surprising. Have I misunderstood what you're trying to show? – Nathaniel Mar 1 '16 at 5:41
• Entropy is only additive in systems without interaction, so your assumption is wrong, already. – CuriousOne Mar 1 '16 at 6:53
• @Nathaniel if I assume P(S)=P(A)P(B) (I don't know if it's a correct statement) then entropy cant be additive because $-P(S)lnP(A)-P(S)lnP(B) \neq -P(A)lnP(A)-P(B)lnP(B)$. That's what I tried to say. – SaudiBombsYemen Mar 1 '16 at 18:27
• @SaudiBombsYemen ah, I see. But when you sum over A and B, they are equal after all. (It that isn't clear, let me know and I'll post an answer.) – Nathaniel Mar 2 '16 at 3:46
• @Nathaniel hmm... I think the problem is that my level of statistics and calculus is still highschool one (last year) so yeah, it'd be perfect if you explained me what you mean :) – SaudiBombsYemen Mar 2 '16 at 15:47

If you consider a system $C$ and two subsystems $A,B$ with associated probability distributions $p_C,p_A,p_B$ and want the entropy to add, you must assume the subsystems are independent in the sense that $p_C(X) = p_A(Y)p_B(Z)$ where $Y$ and $Z$ are a partitioning of the variables $X$ belonging to $C$. Then, the total entropy is \begin{align} S_C & = \langle -\ln(p_C)\rangle\\& = -\int p_C(X)\ln(p_C(X))\mathrm{d}X \\ & = -\int p_A(Y)p_B(Z)(\ln(p_A(Y))+\ln(p_B(Z))\mathrm{d}Y\mathrm{d}Z \\ & = -\int p_A(Y)p_B(Z)\ln(p_A(Y))\mathrm{d}Y\mathrm{d}Z - \int p_A(Y)p_B(Z)\ln(p_B(Z)\mathrm{d}Y\mathrm{d}Z \\ & = S_A+ S_B \end{align} because $\int p_A(Y)\mathrm{d}Y = 1$ and likewise for $B,Z$.
• @ACuriousMind first you wrote $S_c$ as integral of $p_c(X)ln(p_c(X))dX$, that is in terms of X, in the next step, there is no involvement of dZ, and in the step after that there is an involvement of day and dZ, these three steps, I am not able to understand – SK Dash Dec 10 '19 at 7:28