Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Although it seems simple, I can't get the derivation correct. Here is my reasoning:

$P(S)=P(A)P(B)$

Where P is the probability and S, A, and B denote different systems.

$S_A=-P(A)\ln P(A)$ and $S_B=-P(B)\ln P(B)$

Then $$S_{S}=-P(S)\ln P(S)=-P(S)\ln P(A)P(B)=-P(S)\ln P(A)-P(S)\ln P(B)$$

The problem is, since $P(A)P(B)\neq P(S)$, how can entropy be additive?

share|cite|improve this question
3  
I'm pretty confused, because it looks like you start by assuming that $P(S)=P(A)P(B)$ and then show that it contradicts an assumption that $P(S)\ne P(A)P(B)$, which of course shouldn't be surprising. Have I misunderstood what you're trying to show? – Nathaniel Mar 1 at 5:41
    
Entropy is only additive in systems without interaction, so your assumption is wrong, already. – CuriousOne Mar 1 at 6:53
    
@Nathaniel if I assume P(S)=P(A)P(B) (I don't know if it's a correct statement) then entropy cant be additive because $-P(S)lnP(A)-P(S)lnP(B) \neq -P(A)lnP(A)-P(B)lnP(B) $. That's what I tried to say. – SaudiBombsYemen Mar 1 at 18:27
    
@SaudiBombsYemen ah, I see. But when you sum over A and B, they are equal after all. (It that isn't clear, let me know and I'll post an answer.) – Nathaniel Mar 2 at 3:46
    
@Nathaniel hmm... I think the problem is that my level of statistics and calculus is still highschool one (last year) so yeah, it'd be perfect if you explained me what you mean :) – SaudiBombsYemen Mar 2 at 15:47
up vote 16 down vote accepted

If you consider a system $C$ and two subsystems $A,B$ with associated probability distributions $p_C,p_A,p_B$ and want the entropy to add, you must assume the subsystems are independent in the sense that $p_C(X) = p_A(Y)p_B(Z)$ where $Y$ and $Z$ are a partitioning of the variables $X$ belonging to $C$. Then, the total entropy is \begin{align} S_C & = \langle -\ln(p_C)\rangle\\& = -\int p_C(X)\ln(p_C(X))\mathrm{d}X \\ & = -\int p_A(Y)p_B(Z)(\ln(p_A(Y))+\ln(p_B(Z))\mathrm{d}Y\mathrm{d}Z \\ & = -\int p_A(Y)p_B(Z)\ln(p_A(Y))\mathrm{d}Y\mathrm{d}Z - \int p_A(Y)p_B(Z)\ln(p_B(Z)\mathrm{d}Y\mathrm{d}Z \\ & = S_A+ S_B \end{align} because $\int p_A(Y)\mathrm{d}Y = 1$ and likewise for $B,Z$.

The assumption of statistical independence is crucial. Entropy is, from the information-theoretic viewpoint, the expected amount of information encoded in a system. If two systems are not independent, obviously the information encoded in them together will be less than the sum of the information you can extract from each without knowing the other. That is, classical entropy is subadditive, but only additive if the systems you are adding together are statistically independent.

share|cite|improve this answer

Let the two systems be $A$ and $B$ having energy in total $E_{A_i}$ and $E_{B_i}$ respectively.

When they interact with each other, each of the system's energy fluctuates, but the energy of the total system is always constant viz. $E_A + E'_B= E^*= \textrm{const.}$

Now, consider the total system in equilibrium.

What is the probability that $A$ has energy $E_{A}\;?$

Let the total number of accessible microstates of the total system be $\Omega_\textrm{tot}= C^{-1}$.

Now, the probability that $A$ has $E_A$ is given by $$P(E_A)= \frac{\Omega(E_A)\Omega'(E^* - E_A)}{\Omega_\textrm{tot}}$$ where \begin{align}\Omega &= \textrm{number of microstates accessible to}\,\, A\\ \Omega' &= \textrm{number of microstates accessible to}\,\, B\end{align}

Taking logarithm of both sides, we get \begin{align}\ln P(E_A)& = \ln C + \ln \Omega(E_A) + \ln\Omega'(E'_B)\end{align}

Now, since, the equilibrium state is dominated by the macrostate having the maximum number of multiplicities, the expected value of the energy must follow this condition $$\frac{\partial \ln P}{\partial E_A}= 0\;.$$

This gives $$\frac{\partial \ln \Omega}{\partial E_A}= \frac{\partial \ln \Omega'}{\partial E_A}\,\,\,\,\, [\textrm{using}\,\,\, E'_B = E^*- E_A]\;. $$

Each term above is called $\beta$ parameter.

Since, $\beta$ has the dimension of reciprocal energy, we can express it as $$\beta \equiv \frac{1}{k\,T}$$ where \begin{align}k &= \textrm{a positive constant}\\ T &= \textrm{a measure of energy}\;.\end{align}

Therefore, \begin{align}\beta &= \frac{\partial \ln \Omega}{\partial E_A}\\ \frac{1}{T} &\equiv \frac{\partial k \ln \Omega}{\partial E_A}\\ & = \frac{\partial S}{\partial E_A} \end{align} where $$S=k \ln \Omega \;.$$

Now, \begin{align}k\ln\left\{ \frac{P(E_A)}{C}\right\}& = k\ln \Omega(E_A) + k\ln\Omega'(E'_B)\\ \implies k\ln \Omega^*&= k\ln\Omega + k\ln\Omega'\\ S^* & = S_A + S_B\;.\end{align}

share|cite|improve this answer
    
The question does not specify that the system is in equilibrium, and phrases the entropy in probabilistic terms, not in terms of "microstates". – ACuriousMind Feb 29 at 16:58
    
The point is that the entropy in terms of the probabilities coincides with the entropy in terms of the number of microstates (or the phase space volume) only in equilibrium. – ACuriousMind Feb 29 at 17:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.