Take the 2-minute tour ×
Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free, no registration required.

I am looking for a derivation of the formula $$S~=~-\Sigma_ip_i \log (p_i).$$ for entropy, from first principles. I only wish to assume the laws of physics, and without involving concepts in information theory. (After all, the concept of entropy and Boltzmann's formula for it is far older than information theory.)

What is a good definition of entropy? What assumptions are needed to arrive at this? What is the justification to maximizing entropy of a system to arrive at thermodynamics?

share|improve this question

1 Answer 1

You have to assume a couple of things, but first you have to analyze the the system in terms of the phase space. Firts divide the phase space of your system in $k$ cells. Let us assume that your system consist in $N$ particles, where $k\ll N$. We're going to say that a microstate of the system is given by the positions of every single particle. Now, we're gonig to say that the mesostate of the system is given by $\{n_{\alpha}(t)\}\equiv(n_1(t),...,n_N(t))$, where $n_i(t)$ is the average number of particles in the $i$-th cell. And the last thing is this: we are gonig to call the number $W(\{n_{\alpha}(t)\})$ as the number of microsates compatible with the mesostate $\{n_{\alpha}(t)\}$.

Now if $P(\{n_{\alpha}(t)\})$ is the probability of finding the system in the mesostate $\{n_{\alpha}(t)\}$, then we will have

$$P(\{n_{\alpha}(t)\})=W(\{n_{\alpha}(t)\})\cdot p$$ where $p $ is the probability of finding the system un a given microstate.

And now, there are 5 hypotheses that Boltzman assumed to be true:

  1. All the microstates are equally likely, i.e.

$$p=\frac{1}{k^{N}}$$ 2. The system evolves from mesostates of lower probability to mesostates of higher probability. $$P(\{n_{\alpha}(t+\tau)\})\geq P(\{n_{\alpha}(t)\}),\qquad \forall\tau>0$$ 3. The The thermodynamic equilibrium state corresponds to the most probable mesostate. If $\{\tilde n_{\alpha}(t)\}$is the most probable mesostate, then $P(\{\tilde n_{\alpha}(t)\})>P(\{n_{\alpha}(t)\}),\qquad\forall\alpha$.

4 There is a relationship between the entropy of the system and the probability of mesostates. $$S=S(W\{n_{\alpha}(t)\})$$ That's all what you need, for derive that relation.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.