Skip to main content
replaced http://physics.stackexchange.com/ with https://physics.stackexchange.com/
Source Link

Intuitively, this makes entropies add when we calculate the number of microstates for the combination of two systems: if the two systems are independent (uncorrelated), we simply multiply these numbers together.

Secondly, this is the unique definition that makes the quantity equal to the Clausius definition of entropy, namely as the function of state for which $t^{-1} = \frac{\partial S}{\partial Q}$.See my answer hereSee my answer here for a sketch of a proof of this assertion.

Thirdly, and related to my first point above, the logarithm definition gives the quantity an information theoretic meaning as the Shannon information (entropy), i.e. the minimum number of bits (or proportional thereto) needed to describe the microscopic state of a thermalized system given knowledge of its macrostate. See my other answer linked above for further information. This state of affairs is proven by Shannon's Noiseless Coding Theorem if a multiparticle's constituent particles can be taken to be statistically independent.

For example, in the appendix of the seminal paper by E. T. Jaynes:

E. T. Jaynes, "Information Theory and Statistical Mechanics".

sketches Shannon's proof that the entropy, as you;ve defined it, is the unique function, modulo a positive scaling constant, with the three properties that (1) it is always positive, (2) it is a monotonically increasing function of uncertainty about a system's microstate (where the notion of "uncertainty" here is extremely broad and not needfully the multiplicty) and (3) linearly additive for independent sources of uncertainty. The proof is also given in the early chapters of the book Shannon and Weaver, "A Mathemetical Theory of Communication".

Intuitively, this makes entropies add when we calculate the number of microstates for the combination of two systems: if the two systems are independent (uncorrelated), we simply multiply these numbers together.

Secondly, this is the unique definition that makes the quantity equal to the Clausius definition of entropy, namely as the function of state for which $t^{-1} = \frac{\partial S}{\partial Q}$.See my answer here for a sketch of a proof of this assertion.

Thirdly, and related to my first point above, the logarithm definition gives the quantity an information theoretic meaning as the Shannon information (entropy), i.e. the minimum number of bits (or proportional thereto) needed to describe the microscopic state of a thermalized system given knowledge of its macrostate. See my other answer linked above for further information. This state of affairs is proven by Shannon's Noiseless Coding Theorem if a multiparticle's constituent particles can be taken to be statistically independent.

For example, in the appendix of the seminal paper by E. T. Jaynes:

E. T. Jaynes, "Information Theory and Statistical Mechanics".

sketches Shannon's proof that the entropy, as you;ve defined it, is the unique function, modulo a positive scaling constant, with the three properties that (1) it is always positive, (2) it is a monotonically increasing function of uncertainty about a system's microstate (where the notion of "uncertainty" here is extremely broad and not needfully the multiplicty) and (3) linearly additive for independent sources of uncertainty. The proof is also given in the early chapters of the book Shannon and Weaver, "A Mathemetical Theory of Communication".

Intuitively, this makes entropies add when we calculate the number of microstates for the combination of two systems: if the two systems are independent (uncorrelated), we simply multiply these numbers together.

Secondly, this is the unique definition that makes the quantity equal to the Clausius definition of entropy, namely as the function of state for which $t^{-1} = \frac{\partial S}{\partial Q}$.See my answer here for a sketch of a proof of this assertion.

Thirdly, and related to my first point above, the logarithm definition gives the quantity an information theoretic meaning as the Shannon information (entropy), i.e. the minimum number of bits (or proportional thereto) needed to describe the microscopic state of a thermalized system given knowledge of its macrostate. See my other answer linked above for further information. This state of affairs is proven by Shannon's Noiseless Coding Theorem if a multiparticle's constituent particles can be taken to be statistically independent.

For example, in the appendix of the seminal paper by E. T. Jaynes:

E. T. Jaynes, "Information Theory and Statistical Mechanics".

sketches Shannon's proof that the entropy, as you;ve defined it, is the unique function, modulo a positive scaling constant, with the three properties that (1) it is always positive, (2) it is a monotonically increasing function of uncertainty about a system's microstate (where the notion of "uncertainty" here is extremely broad and not needfully the multiplicty) and (3) linearly additive for independent sources of uncertainty. The proof is also given in the early chapters of the book Shannon and Weaver, "A Mathemetical Theory of Communication".

Source Link
Selene Routley
  • 89.3k
  • 7
  • 195
  • 411

Intuitively, this makes entropies add when we calculate the number of microstates for the combination of two systems: if the two systems are independent (uncorrelated), we simply multiply these numbers together.

Secondly, this is the unique definition that makes the quantity equal to the Clausius definition of entropy, namely as the function of state for which $t^{-1} = \frac{\partial S}{\partial Q}$.See my answer here for a sketch of a proof of this assertion.

Thirdly, and related to my first point above, the logarithm definition gives the quantity an information theoretic meaning as the Shannon information (entropy), i.e. the minimum number of bits (or proportional thereto) needed to describe the microscopic state of a thermalized system given knowledge of its macrostate. See my other answer linked above for further information. This state of affairs is proven by Shannon's Noiseless Coding Theorem if a multiparticle's constituent particles can be taken to be statistically independent.

For example, in the appendix of the seminal paper by E. T. Jaynes:

E. T. Jaynes, "Information Theory and Statistical Mechanics".

sketches Shannon's proof that the entropy, as you;ve defined it, is the unique function, modulo a positive scaling constant, with the three properties that (1) it is always positive, (2) it is a monotonically increasing function of uncertainty about a system's microstate (where the notion of "uncertainty" here is extremely broad and not needfully the multiplicty) and (3) linearly additive for independent sources of uncertainty. The proof is also given in the early chapters of the book Shannon and Weaver, "A Mathemetical Theory of Communication".