Entropy definition, additivity, laws in different ensembles

Question

I started studying Statistical Mechanics and specifically - microcanonical ensemble, which consists of defining different quantities such as temperature and entropy, and deriving some thermodynamic relations. After I finished it, I went to canonical ensemble and then many questions came to my mind. I will ask a lot of them and if it is wrong then please tell me and I can divide them in different posts.

For you to better understand what concerns me, I am going to write questions together with a thoughts I have about them.

In microcanonical ensemble, many books prefer to consider an isolated system with some energy and then divide this system into two systems (system 1 and system 2) that are separated by a diathermal wall - ones that can exchange energy by heat.

1. If energy of the isolated system is $E$, volume is $V$ and number of particles is $N$, then entropy $S$ is defined as $S(E,V,N) = k_{B}\ln{\Omega(E,V,N)}$. Here, $k_{B}$ is Boltzmann's constant and ${\Omega}$ is the number of microstates that correspond to the specific macroscopic parameters mentioned previously. For which of the three systems is entropy defined this way?

I am pretty sure that entropy is defined in such manner for the two systems separated by a diathermal wall, but I am very unsure about the isolated system. By physical considerations I would assume that such definition applies for the isolated system by saying that no system is better than other. On the other hand, if I assume that this is true then it leads me to a contradiction in further questions.

2. Why entropy of an isolated system is equal to the sum of subsystems $S=S_1+S_2$ ?

I think I can derive this if subsystems are isolated from each other and I think I can derive (as an approximation) it when both subsystems are in equilibrium. But I have no idea about transition to equilibrium. Also I am not sure whether it should be that way at all - because macroscopic parameters of an isolated system do not change - entropy should not change too. But it is "derived" somehow that entropy of the isolated system increases. Isn't it a contradiction? On the other hand, if I assume that $S$ is entropy of the isolated system when I know that subsystem 1 has, for example, energy $E_1$, volume $V_1$ and number of particles $N_1$, then I understand this. But then shouldn't this mean that to know entropy of a system I have to know everything about it's internal construction - each subsystem which can then be divided in more subsystems etc.

3. Why $dS_{isolated system} \geq 0$?

I am aware that this is an experimental fact and it has a probabilistic nature and also that for some systems it does not hold. Is there a theoretical model that can motivate this equation? Or maybe if there is a number of conditions (known from experiments, for example) that must hold for this to be true?

4. Is temperature, pressure, chemical potential etc. which are defined as derivated of entropy with respect to something defined for non-equilibrium also?

From high school studies I am pretty sure (which means I don't know) that pressure and temperature should be defined even when subsystems interact and they are not in equilibrium. But if that is true - both average energy and entropy should be defined in non-equilibrium. I have no problems with average energy just as it has no problems with me. But what about entropy? I understand why we use entropy formula that connects it to the microstates in equilibrium (assuming dominance of one macrostate). But what about non-equilibrium? I don't even know what energy subsystems have at each instant of time.

REMARK : I know answers to all of the questions if I assume that systems consist of identically distributed particles and that we take limit as number of particles in each subsystem tends to infinity and then approximate that in equilibrium energy $E$ density distribution for each of the systems is $\delta(E-C)$, where $C$ depends on the system. And I thought this is what happens - but then I started reading about canonical ensemble ... Because we assume one system is microscopic, so previous arguments can not be used (number of particles can be one which is not even close to infinity).

5. In canonical ensemble - energy distribution of subsystems is not delta distribution - therefore we talk about average energies. But then how and why is entropy defined?

Do I take into account only microstates that have energy equal to the average energy? But then different distributions of energies could have the same entropy which is non-intuitive to me. On the other hand, if it is another function - what is it? And why is it the way it is? For me it would make sense to define entropy of a system to be something like an average of entropy for each of the energy macrostates. Because then I can calculate it and distribution of energy matters. But then, how is it connected to average energy? Because I surely have to calculate derivative with respect to it.

TL;DR: What is entropy? What is the meaning of life?

I thank everyone that has read my post and helped me!

Answer 1

1. It is defined in that way for the system which has the macrostates $(N,E,V)$, i.e. for the isolated system.

Note though, that this $\Omega$ isn't that straightforward. For the system 1 and system 2, suppose you have number of particles $N_1,\, N_2$ and energies $E_1,\, E_2$, respectively, constrained to satisfy $N_1+N_2=N$ and $E_1+E_2=E$. Before the exchange of energy you had (if $S_1,\,S_2$ and $S$ are respectively the entropies of system 1, system 2 and isolated system) \begin{align*} S_1^{(0)} &= k_{B} \log\left[\Omega_1\left(E_1^{(0)}\right)\right] \\ S_2^{(0)} &= k_{B} \log\left[\Omega_2\left(E_2^{(0)}\right)\right], \end{align*} where the parenthesized superscript $(0)$ denotes it is the initial state. So initially $$\Omega^{(0)} (E) = \Omega_1\left(E_1^{(0)}\right)\;\Omega_2\left(E_1^{(0)}\right), $$ But after exchanging energy it is $$\Omega(E) = \sum_{E_1} \Omega_1(E_1)\;\Omega_2(E-E_1).$$ Thus if $ E_1^{\star} := \mathrm{argmax}(\Omega)$, i.e. if $ E_1^{\star} := \bigg\{ E_1 : \Omega\left(E_1^{\star}\right) = \max\Omega(E) \bigg\} $ and $E_2^{\star} := E-E_1^{\star}$, because the function $\Omega_1\Omega_2$ as a function of $E_1$ has the form of a very very steep bell curve with only one maximum at $E_1^\star$ (which we'll denote $\Omega_1^\star\Omega_2^\star$), you get [and this gets us to ...]

2. $$ S \simeq k_B\log\Omega_1^\star + k_B\log\Omega_2^\star. $$

But in reality it is only $ k_B\log\Omega_1^\star + k_B\log\Omega_2^\star \leqslant S \leqslant k_B\log\Omega_1^\star + k_B\log\Omega_2^\star + k_B\log{\mathcal{M}},$ where $\mathcal{M}$ is the number of terms in the sum that defines the evolution of $\Omega$. But this is $\mathcal{M} = \mathcal{O}(N)$ (meaning of the order of $N$), thus $\log{\mathcal{M}} = \mathcal{O}\left(\log{N}\right) \ll \mathcal{O}(N)$, which we want $S$ to be. That's why throwing the $\mathcal{M}$-term away is justified. But if you want to have 1 particle as you said (or generally not-too-many particles) you can't throw it away and you have to keep the inequality as the best you can work with. Going into

3. is easy. You can calculate \begin{align*} \Delta S &= S - S^{(0)} =^{(\star)} k_B\log\Omega_1^\star + k_B\log\Omega_2^\star - k_B\log\Omega_1^{(0)} + k_B\log\Omega_2^{(0)} = k_B\log\left( \frac{\Omega_1^\star\Omega_2^\star}{\Omega_1^{(0)}\Omega_2^{(0)}} \right) \\ &\geqslant k_B\log\left( \frac{\Omega_1^{(0)}\Omega_2^{(0)}}{\Omega_1^{(0)}\Omega_2^{(0)}} \right) = 0. \end{align*} ${}^{(\star)}$ if you keep the $\mathcal{Μ}$-term the equal sign becomes greater-or-equal, so either ways $\Delta S\geqslant 0$ is correct.

4. The easy answer is $$ S := -\frac{\partial F}{\partial T}, $$ where $F$ is Helmholtz free energy. But that'd be a circular argument. The actual answer is to consider an isolated system "t", containing system "s" (for small) and "r" (for rest). In there you have $E_t = E_s + E_r$, but now you let them exchange temperature (whatever this beast is). Then the entropy is $$ S := -k_B \sum_{\{\vec{\mu}_s\}} \Pr(\vec{\mu}_s)\log\left(\Pr(\vec{\mu}_s)\right), $$ (Shanon's definition). But this is a constrained sum and its calculation is far from trivial. The incorrect, circular definition with the free energy turns out much more useful at the end of the day. The connection with the average energy comes from the Helmholtz-beast again, because, $\left\langle E \right\rangle = \partial_{\beta}(\beta F)$ and $S = -\partial_{T}(F) = \cdots = k_B \beta^2\partial_{\beta}(F)$.

EDIT: I don't know what the meaning of life is, but, frankly I don't care. Generalizing things to more and more abstract levels is fun maaaan.

Answer 2

1. Entropy is defined as S=kln($ \Omega $) always. Whether it is a function of (N,V,E) or (N,V,T) or ($ \mu $,V,T) depends on the ensemble.

2. For an isolated system the entropy is a sum ONLY (or at least generally, maybe there is some example that I'm missing) when the subsystems are noninteracting.

For example: How many ways are there of creating a password using only numbers that is 2 characters long?

Answer: Well, it is 10*10 since there are 10 options for the first slot and 10 options for the second slot.

A similar thing holds for noninteracting systems. If subsystem 1 has X ways of arranging and subsystem 2 has Y ways of arranging, then the total system has X*Y ways of arranging itself.

Thus the entropy of this noninteracting system which is made of system 1 and system 2 has entropy:

S(total)= kln(X*Y) = kln(X) +kln(Y) = S(1) +S(2)

In other words, the entropy adds in this case.

3. This is just basically an assumption of statistical mechanics. For example: How many ways are there of arranging two coins:

(H | H) OR (H | T) OR (T | H) OR (T | T)

Thus, the most probable outcome is that 1 coin lands on H and 1 coin lands on T. In other words, the MOST LIKELY outcome is the one that has the largest $ \Omega $.

We generalize this for physical systems. The system can be is some arbitrary arrangement that agrees with the constraints of energy and temperature etc.... But we assume that it will be in the MOST LIKELY arrangement, in other words the arrangement that has the largest $ \Omega $. Thus, if the system is in a state where $ \Omega $ is NOT maximal, we assume that over time it will become the maximal one. Therefore, $ \Omega $ is assumed to either stay the same (already maximum) or increase to the maximum value.

Since S = kln($ \Omega $), if $ \Omega $ always is assumed to go to the largest $ \Omega $ possible, S (the entropy) will always increase.

4. To be honest, these kind of follow from definition. We DEFINE all these quantities as derivatives (or some other combination) of the entropy S. It just so happens that we chose the definitions to agree with classical ideas. But it is better to think of it as definitions, I think.

5. Again, entropy can be a function of (N,V,E) or (N,V,T) or whatever. It is still well defined. We defined it for the canonical ensemble because often times energy ISN'T what is easily controlled. Usually we fix temperature, and the energy can vary. So then S=S(N,V,T) is a function of temperature, not energy.