Take the 2-minute tour ×
Physics Stack Exchange is a question and answer site for active researchers, academics and students of physics. It's 100% free, no registration required.

Possible Duplicate:
Proof of $S=-\sum p\ln p$?

I am looking for a derivation of the formula $$S~=~-\Sigma_ip_i \log (p_i).$$ for entropy, from first principles. I only wish to assume the laws of physics, and without involving concepts in information theory. (After all, the concept of entropy and Boltzmann's formula for it is far older than information theory.)

What is a good definition of entropy? What assumptions are needed to arrive at this? What is the justification to maximizing entropy of a system to arrive at thermodynamics?

share|improve this question

marked as duplicate by Qmechanic, David Z Nov 20 '12 at 2:46

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Possible duplicate: physics.stackexchange.com/q/14436/2451 –  Qmechanic Nov 19 '12 at 23:51
@Qmechanic I am looking for a discussion on entropy that is purely based on physics I will modify the question to reflect so. The thread Derives the formula from Information theory, which is not what I am interested in. –  Prathyush Nov 20 '12 at 4:48
@Prathyush if you are not interested in information theory then you are not interested in finding the answer to your question!! The first-principles justification for maximising entropy (and the justification for the formula you give) is that we are clumsy monkeys doing the best we can with our limited experimental resolution: it has little to do with fundamental properties or laws of the universe. If you read the other thread you would find that this was all explained by Jaynes in the 50s, e.g. see the original paper here. It's a must read. –  Mark Mitchison Nov 20 '12 at 5:20
@MarkMitchison Thanks for pointing to Jaynes paper. While I agree we are clumsy monkey, and this is best our experimental arrangements can give us, I want ask why are we clumsy in that precise manner. Let me read the paper first though. –  Prathyush Nov 20 '12 at 5:45
"The principle of maximum entropy can be regarded as an extension to the principle of insufficient reason of Laplace(to which it reduces in case no information is given except the enumeration of the possibilities x_i) with the following essential difference. The maximum-entropy distribution may be asserted for the positive reason that it is uniquely determined by the non committal with regard to the missing information, instead of the negative one that there is no reason to think otherwise. " –  Prathyush Nov 20 '12 at 5:59

1 Answer 1

You have to assume a couple of things, but first you have to analyze the the system in terms of the phase space. Firts divide the phase space of your system in $k$ cells. Let us assume that your system consist in $N$ particles, where $k\ll N$. We're going to say that a microstate of the system is given by the positions of every single particle. Now, we're gonig to say that the mesostate of the system is given by $\{n_{\alpha}(t)\}\equiv(n_1(t),...,n_N(t))$, where $n_i(t)$ is the average number of particles in the $i$-th cell. And the last thing is this: we are gonig to call the number $W(\{n_{\alpha}(t)\})$ as the number of microsates compatible with the mesostate $\{n_{\alpha}(t)\}$.

Now if $P(\{n_{\alpha}(t)\})$ is the probability of finding the system in the mesostate $\{n_{\alpha}(t)\}$, then we will have

$$P(\{n_{\alpha}(t)\})=W(\{n_{\alpha}(t)\})\cdot p$$ where $p $ is the probability of finding the system un a given microstate.

And now, there are 5 hypotheses that Boltzman assumed to be true:

  1. All the microstates are equally likely, i.e.

$$p=\frac{1}{k^{N}}$$ 2. The system evolves from mesostates of lower probability to mesostates of higher probability. $$P(\{n_{\alpha}(t+\tau)\})\geq P(\{n_{\alpha}(t)\}),\qquad \forall\tau>0$$ 3. The The thermodynamic equilibrium state corresponds to the most probable mesostate. If $\{\tilde n_{\alpha}(t)\}$is the most probable mesostate, then $P(\{\tilde n_{\alpha}(t)\})>P(\{n_{\alpha}(t)\}),\qquad\forall\alpha$.

4 There is a relationship between the entropy of the system and the probability of mesostates. $$S=S(W\{n_{\alpha}(t)\})$$ That's all what you need, for derive that relation.

share|improve this answer
I did not understand the last point, Please can you elaborate. What is the entropy defined as in the first place. I have modified the question a little bit. –  Prathyush Nov 20 '12 at 4:59
@Prathyush try googling 'Boltzmann's tombstone' ;) –  Mark Mitchison Nov 20 '12 at 5:23
Entropy –  Anuar Nov 20 '12 at 5:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.