Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It only takes a minute to sign up.
Sign up to join this community
I would like to ask if someone knows the physical meaning of Boltzmann's definition of entropy. Of course the formula is pretty straightforward
$$S=k_b\ln(Ω)$$
but what in the heck is the natural logarithm of microstates of the systems? I mean what does it express?
Two obvious desirable features of this definition are:
This relation can be obtained form the assumption of equal a priori probabilities, i.e. that the equilibrium corresponds to the macrostate with the maximum number of microstates:
Consider two isolated systems, separately in equilibrium, each with macrostates $E_i^{(0)}, V_i, N_i$ (energy, volume, number of particles). Each of them has a total number of $\Omega_i(N_i,V_i,E_i^{(0)})$ possible microstates.
Now we bring them into thermal contact so that they can exchange energy. After this point, we'll have $E_t=E_1'+E_2'=\text{constant}$. $N$ and $V$ for each system remain unchanged. The total number of possible microstates for each system would be $\Omega_i(N_i,V_i,E_i')$ and for the composite system: $$\Omega=\Omega_1(N_1,V_1,E_1')\times \Omega_2(N_2,V_2,E_2')=\Omega_1(E_1')\Omega_2(E_2')=$$ $$\Omega(E_t,E_1)$$
With the assumption of equilibrium occurs at the point of having maximum $\Omega$, we find the value of $E_1^*$ (and hence $E_2^*$) that maximizes $\Omega(E_t,E_1)$: $$d\Omega=0\to\left (\frac{\partial\Omega_1(E_1)}{\partial E_1}\right )_{E_1=E_1^*} \Omega_2(E_2^*) +\Omega_1(E_1^*)\left (\frac{\partial\Omega_2(E_2)}{\partial E_2}\right )_{E_2=E_2^*}\frac{\partial E_2}{\partial E_1}=0\tag{1}$$ $$\frac{\partial E_2}{\partial E_1}=-1\to$$ $$\beta_1=\left (\frac{\partial \ln \Omega_1(E_1)}{\partial E_1}\right )_{E_1=E_1^*}=\left (\frac{\partial \ln \Omega_2(E_2)}{\partial E_2}\right )_{E_2=E_2^*}=\beta_2\tag{2}$$
Naturally we expect these quantities $\beta_1$ and $\beta_2$ to be related to temperatures of the systems. From thermodynamics we know that $$\left(\frac{\partial S}{\partial E}\right)_{N,V}=\frac{1}{T}\tag{3}$$ Comparing $(2)$ and $(3)$, we can conclude that: $$\frac{\partial S}{\partial (\ln \Omega)}=k$$ or $$\Delta S=k\ln \Omega$$ where $k$ is a constant.
Mustafa's answer gives one important reason for the logarithmic dependence: microstates multiply, whereas we'd like an extrinsic property of a system to be additive. So we simply need an isomorphism that turns multiplication into addition. The only continuous one is the "slide rule isomorphism" aka the logarithm. The base $e$ is arbitrary as you can see from Mustafa's answer: you can use any positive base (aside from 1!), and, as you shift base, you will need to adjust the Boltzmann constant $k_B$ to absorb the multiplicative change-of-base factor.
But an information-theoretic look at the number of possible microstates shows other deep reasons aside from the above. The proof of the Shannon Noiseless Coding Theorem, the gives the informational entropy (also logarithmic) its working meaning: it is the minimum number of bits, or number of "yes-no" answers we need to answer to uniquely identify a particular microstate, assuming all are equally likely. Imagine all the possible microstates arranged in some lexicographic order and then imagine keeping them "on file" in a database arranged as a binary tree. You work your way down the binary tree to find a particular microstate, and the number of branches you need to make on the way (proportional to your seek-and-retrieve time) is $\log_2\Omega$. Or, intuitively, entropy is the length of the shortest book you'd need to write to describe a particular microstate given a system's macroscopic properties. That's some book: if we add merely one joule of heat to a system at one degree kelvin (colder than the cosmic background microwave radiation in deep space), we would need a book bigger than the whole World Wide Web at the end of 2013 to describe the system's microstate!
As I said, you can use $\log_e$ instead of $\log_2$ as long as you keep track of the multiplicative change of base factor in your physical constants (defintion of $k_B$ and $T$).
Section 2 (read it carefully) and the appendix of this paper:
E. T. Jaynes, "Information Theory and Statistical Mechanics".
also give some solid motivations for the logarithm and the entropy formula as being the unique dependence with all the following properties:
If you think about it, the last point (3) is a powerful generalisation of the "multiplication becomes addition" idea expressed in Mustafa's answer.
Entropy was first met in classical thermodynamics and was defined as
$$\mathrm dS= \frac{\delta Q}{T}$$ , where $Q$ comes from the first law of thermodynamics
$$\Delta U= Q- W$$
and $T$ is the temperature; $W$ work done by the system.
Once it was established experimentally that matter at the micro level is discrete, i.e. is composed of molecules the statistical behavior of matter became the underlying framework from which classical thermodynamics emerges.
The first law is conservation of energy, which is also a strict law in the ensembles of the microsystem.
It was established in statistical mechanics that the average kinetic energy of the particles is connected with temperature.
The way the classical entropy emerges and is identified with the statistical mechanics derived entropy is not simple.
The statistical definition was developed by Ludwig Boltzmann in the 1870s by analyzing the statistical behavior of the microscopic components of the system. Boltzmann showed that this definition of entropy was equivalent to the thermodynamic entropy to within a constant number which has since been known as Boltzmann's constant. In summary, the thermodynamic definition of entropy provides the experimental definition of entropy, while the statistical definition of entropy extends the concept, providing an explanation and a deeper understanding of its nature.
This paper, for example, proves ( equation 42) that the statistical mechanics entropy is identified with the entropy of classical thermodynamics. The logarithmic dependence comes from the mathematics of the proof of equivalence.
