# Entropy definition, additivity, laws in different ensembles

I started studying Statistical Mechanics and specifically - microcanonical ensemble, which consists of defining different quantities such as temperature and entropy, and deriving some thermodynamic relations. After I finished it, I went to canonical ensemble and then many questions came to my mind. I will ask a lot of them and if it is wrong then please tell me and I can divide them in different posts.

For you to better understand what concerns me, I am going to write questions together with a thoughts I have about them.

In microcanonical ensemble, many books prefer to consider an isolated system with some energy and then divide this system into two systems (system 1 and system 2) that are separated by a diathermal wall - ones that can exchange energy by heat.

1. If energy of the isolated system is $E$, volume is $V$ and number of particles is $N$, then entropy $S$ is defined as $S(E,V,N) = k_{B}\ln{\Omega(E,V,N)}$. Here, $k_{B}$ is Boltzmann's constant and ${\Omega}$ is the number of microstates that correspond to the specific macroscopic parameters mentioned previously. For which of the three systems is entropy defined this way?

I am pretty sure that entropy is defined in such manner for the two systems separated by a diathermal wall, but I am very unsure about the isolated system. By physical considerations I would assume that such definition applies for the isolated system by saying that no system is better than other. On the other hand, if I assume that this is true then it leads me to a contradiction in further questions.

1. Why entropy of an isolated system is equal to the sum of subsystems $S=S_1+S_2$ ?

I think I can derive this if subsystems are isolated from each other and I think I can derive (as an approximation) it when both subsystems are in equilibrium. But I have no idea about transition to equilibrium. Also I am not sure whether it should be that way at all - because macroscopic parameters of an isolated system do not change - entropy should not change too. But it is "derived" somehow that entropy of the isolated system increases. Isn't it a contradiction? On the other hand, if I assume that $S$ is entropy of the isolated system when I know that subsystem 1 has, for example, energy $E_1$, volume $V_1$ and number of particles $N_1$, then I understand this. But then shouldn't this mean that to know entropy of a system I have to know everything about it's internal construction - each subsystem which can then be divided in more subsystems etc.

1. Why $dS_{isolated system} \geq 0$?

I am aware that this is an experimental fact and it has a probabilistic nature and also that for some systems it does not hold. Is there a theoretical model that can motivate this equation? Or maybe if there is a number of conditions (known from experiments, for example) that must hold for this to be true?

1. Is temperature, pressure, chemical potential etc. which are defined as derivated of entropy with respect to something defined for non-equilibrium also?

From high school studies I am pretty sure (which means I don't know) that pressure and temperature should be defined even when subsystems interact and they are not in equilibrium. But if that is true - both average energy and entropy should be defined in non-equilibrium. I have no problems with average energy just as it has no problems with me. But what about entropy? I understand why we use entropy formula that connects it to the microstates in equilibrium (assuming dominance of one macrostate). But what about non-equilibrium? I don't even know what energy subsystems have at each instant of time.

REMARK : I know answers to all of the questions if I assume that systems consist of identically distributed particles and that we take limit as number of particles in each subsystem tends to infinity and then approximate that in equilibrium energy $E$ density distribution for each of the systems is $\delta(E-C)$, where $C$ depends on the system. And I thought this is what happens - but then I started reading about canonical ensemble ... Because we assume one system is microscopic, so previous arguments can not be used (number of particles can be one which is not even close to infinity).

1. In canonical ensemble - energy distribution of subsystems is not delta distribution - therefore we talk about average energies. But then how and why is entropy defined?

Do I take into account only microstates that have energy equal to the average energy? But then different distributions of energies could have the same entropy which is non-intuitive to me. On the other hand, if it is another function - what is it? And why is it the way it is? For me it would make sense to define entropy of a system to be something like an average of entropy for each of the energy macrostates. Because then I can calculate it and distribution of energy matters. But then, how is it connected to average energy? Because I surely have to calculate derivative with respect to it.

TL;DR: What is entropy? What is the meaning of life?

I thank everyone that has read my post and helped me!

• These are great questions, but there are too many of them in one post. If you don't agree, look at the posts that get answered and how they are structured and focused on one or two points. I strongly suggest you take them one at a time as seperate posts and the best of luck with them. The meaning of life is easy: 42 - (the universe + everything). – user140606 Jan 17 '17 at 22:37
• @Countto10 Thank you for suggestion! And thank you for answering what is the meaning of life - now I don't even need to make any more posts! :) – Daniels Krimans Jan 17 '17 at 22:42

1. It is defined in that way for the system which has the macrostates $(N,E,V)$, i.e. for the isolated system.

Note though, that this $\Omega$ isn't that straightforward. For the system 1 and system 2, suppose you have number of particles $N_1,\, N_2$ and energies $E_1,\, E_2$, respectively, constrained to satisfy $N_1+N_2=N$ and $E_1+E_2=E$. Before the exchange of energy you had (if $S_1,\,S_2$ and $S$ are respectively the entropies of system 1, system 2 and isolated system) \begin{align*} S_1^{(0)} &= k_{B} \log\left[\Omega_1\left(E_1^{(0)}\right)\right] \\ S_2^{(0)} &= k_{B} \log\left[\Omega_2\left(E_2^{(0)}\right)\right], \end{align*} where the parenthesized superscript $(0)$ denotes it is the initial state. So initially $$\Omega^{(0)} (E) = \Omega_1\left(E_1^{(0)}\right)\;\Omega_2\left(E_1^{(0)}\right),$$ But after exchanging energy it is $$\Omega(E) = \sum_{E_1} \Omega_1(E_1)\;\Omega_2(E-E_1).$$ Thus if $E_1^{\star} := \mathrm{argmax}(\Omega)$, i.e. if $E_1^{\star} := \bigg\{ E_1 : \Omega\left(E_1^{\star}\right) = \max\Omega(E) \bigg\}$ and $E_2^{\star} := E-E_1^{\star}$, because the function $\Omega_1\Omega_2$ as a function of $E_1$ has the form of a very very steep bell curve with only one maximum at $E_1^\star$ (which we'll denote $\Omega_1^\star\Omega_2^\star$), you get [and this gets us to ...]

1. $$S \simeq k_B\log\Omega_1^\star + k_B\log\Omega_2^\star.$$

But in reality it is only $k_B\log\Omega_1^\star + k_B\log\Omega_2^\star \leqslant S \leqslant k_B\log\Omega_1^\star + k_B\log\Omega_2^\star + k_B\log{\mathcal{M}},$ where $\mathcal{M}$ is the number of terms in the sum that defines the evolution of $\Omega$. But this is $\mathcal{M} = \mathcal{O}(N)$ (meaning of the order of $N$), thus $\log{\mathcal{M}} = \mathcal{O}\left(\log{N}\right) \ll \mathcal{O}(N)$, which we want $S$ to be. That's why throwing the $\mathcal{M}$-term away is justified. But if you want to have 1 particle as you said (or generally not-too-many particles) you can't throw it away and you have to keep the inequality as the best you can work with. Going into

1. is easy. You can calculate \begin{align*} \Delta S &= S - S^{(0)} =^{(\star)} k_B\log\Omega_1^\star + k_B\log\Omega_2^\star - k_B\log\Omega_1^{(0)} + k_B\log\Omega_2^{(0)} = k_B\log\left( \frac{\Omega_1^\star\Omega_2^\star}{\Omega_1^{(0)}\Omega_2^{(0)}} \right) \\ &\geqslant k_B\log\left( \frac{\Omega_1^{(0)}\Omega_2^{(0)}}{\Omega_1^{(0)}\Omega_2^{(0)}} \right) = 0. \end{align*} ${}^{(\star)}$ if you keep the $\mathcal{Μ}$-term the equal sign becomes greater-or-equal, so either ways $\Delta S\geqslant 0$ is correct.

2. The easy answer is $$S := -\frac{\partial F}{\partial T},$$ where $F$ is Helmholtz free energy. But that'd be a circular argument. The actual answer is to consider an isolated system "t", containing system "s" (for small) and "r" (for rest). In there you have $E_t = E_s + E_r$, but now you let them exchange temperature (whatever this beast is). Then the entropy is $$S := -k_B \sum_{\{\vec{\mu}_s\}} \Pr(\vec{\mu}_s)\log\left(\Pr(\vec{\mu}_s)\right),$$ (Shanon's definition). But this is a constrained sum and its calculation is far from trivial. The incorrect, circular definition with the free energy turns out much more useful at the end of the day. The connection with the average energy comes from the Helmholtz-beast again, because, $\left\langle E \right\rangle = \partial_{\beta}(\beta F)$ and $S = -\partial_{T}(F) = \cdots = k_B \beta^2\partial_{\beta}(F)$.

EDIT: I don't know what the meaning of life is, but, frankly I don't care. Generalizing things to more and more abstract levels is fun maaaan.

• Thanks for answer but unfortunately you answered none of the questions. Comments: for 1. You say that it is defined for an isolated system, then how can entropy of it increase if E,V,N are fixed? 2. Where do you get the inequality from and why would you neglect something that asymptotes as log(N) (for example, in the thermodynamic limit it would go to infinity, wouldn't it?)? 3. This does not prove the assumption that entropy is non-decreasing as time goes on as system goes to equilibrium. 4. I would like to see the connection between entropy defined before. – Daniels Krimans Jan 18 '17 at 17:59
• @DanielsKrimans: his reply is perfectly correct. Note that he never says that $\Omega(E)$ increases. Instead he shows that $\Omega(E)$ is a sum over energy values of system 1. So the log of it cannot be straightforwardly a sum of entropies. – gatsu Jan 18 '17 at 19:30
• I strongly disagree with point 3. The second law of thermodynamics has strictly nothing to do with equilibrium statistical mechanics. – gatsu Jan 18 '17 at 19:35
• @gatsu I did not say that his reply is incorrect (because I do not know). It is just I did not gain anything from it. And he says that $\Omega(E)$ increases - see his point 3. I should probably also rephrase my comments towards point 3: This does not prove the assumption that as the isolated system evolves in time, derivative of $S$ with respect to time is always non-negative. – Daniels Krimans Jan 18 '17 at 19:58
• @DanielsKrimans: I also disagree with his point 3 – gatsu Jan 18 '17 at 20:35
1. Entropy is defined as S=kln($\Omega$) always. Whether it is a function of (N,V,E) or (N,V,T) or ($\mu$,V,T) depends on the ensemble.

2. For an isolated system the entropy is a sum ONLY (or at least generally, maybe there is some example that I'm missing) when the subsystems are noninteracting.

For example: How many ways are there of creating a password using only numbers that is 2 characters long?

Answer: Well, it is 10*10 since there are 10 options for the first slot and 10 options for the second slot.

A similar thing holds for noninteracting systems. If subsystem 1 has X ways of arranging and subsystem 2 has Y ways of arranging, then the total system has X*Y ways of arranging itself.

Thus the entropy of this noninteracting system which is made of system 1 and system 2 has entropy:

S(total)= kln(X*Y) = kln(X) +kln(Y) = S(1) +S(2)

In other words, the entropy adds in this case.

1. This is just basically an assumption of statistical mechanics. For example: How many ways are there of arranging two coins:

(H | H) OR (H | T) OR (T | H) OR (T | T)

Thus, the most probable outcome is that 1 coin lands on H and 1 coin lands on T. In other words, the MOST LIKELY outcome is the one that has the largest $\Omega$.

We generalize this for physical systems. The system can be is some arbitrary arrangement that agrees with the constraints of energy and temperature etc.... But we assume that it will be in the MOST LIKELY arrangement, in other words the arrangement that has the largest $\Omega$. Thus, if the system is in a state where $\Omega$ is NOT maximal, we assume that over time it will become the maximal one. Therefore, $\Omega$ is assumed to either stay the same (already maximum) or increase to the maximum value.

Since S = kln($\Omega$), if $\Omega$ always is assumed to go to the largest $\Omega$ possible, S (the entropy) will always increase.

1. To be honest, these kind of follow from definition. We DEFINE all these quantities as derivatives (or some other combination) of the entropy S. It just so happens that we chose the definitions to agree with classical ideas. But it is better to think of it as definitions, I think.

2. Again, entropy can be a function of (N,V,E) or (N,V,T) or whatever. It is still well defined. We defined it for the canonical ensemble because often times energy ISN'T what is easily controlled. Usually we fix temperature, and the energy can vary. So then S=S(N,V,T) is a function of temperature, not energy.

• Thanks for answer. My comments and questions: 1. If it is defined like that always - then how can it be possible for an isolated system to increase it's number of macrostates (and therefore entropy)? 2. $S(total) = S(1)+S(2)$ surely does not follow from logarithm properties as it would mean that subsystem 1 can have only one value of energy - which is not true because it is not isolated and can interact! 3. Thanks! Maybe some conditions to know when it holds? 4. So are they defined in non-equilibrium or not? 5. I agree that it is defined - but how and why? – Daniels Krimans Jan 18 '17 at 20:12
• 1. Because $\Omega =\Omega(N,V,T)$ is a function of other variables. For example, simply saying "you have 2 coins" doesn't tell you if it is (H|H) or (H|T) or what. Thus if you are only given N=2, then your $\Omega = \Omega(N)$ is a function of N. Only given the information N=2 doesn't tell you which state you are in. Similar things follow for (N,V,T) and the other ones. There are many ways of arranging the system so that N,V,T are the SAME for every arrangement. You want to pick out the arrangement with the MOST ways of doing so.... – Ben S Jan 18 '17 at 23:23
• 1. The system does NOT increase it's macrostates. It's macrostates are what they are. The system increases its entropy by going to the state with the most MICROstates. – Ben S Jan 18 '17 at 23:31
• 4. Yes, pressure has a definition even in nonequilibrium. 5. It's defined in exactly the same way. You ask how many different ways there for a system to be arranged in, given that it's total particles are N, its volume is V, and its temperature is T. – Ben S Jan 18 '17 at 23:34
• I was too tired when I wrote comment - ofcourse I meant microstates not macrostates. Why would number of microstates increase if system is isolated? Thanks about other answers - I think I got it. – Daniels Krimans Jan 19 '17 at 13:03