Why is entropy's definition useful?

Question

I have somewhat of an understanding for other physical quantities, but as far as entropy goes I only know it to be "disorder". Why is the change in entropy formula an appropriate/useful definition, moreover, why is the equation for entropy an appropriate statement for entropy. With things like volume and pressure, at least, I have a natural inkling as to what they are. What is entropy, more than disorder. Why are there no units? Why are these definitions correct?

Answer 1

What is entropy, more than disorder.

Mathematically, entropy is just a measure of spread of a probability distribution: The lower the entropy, the more spiked the distribution.

In statistical mechanics, a state is generally only partially defined via some macroscopic constraints, and entropy is a measure of microscopic indeterminacy. Maximizing entropy means choosing the least biased probability distribution compatible with the available information.

In equilibrium thermodynamics, we go from statistical ensemble averages to thermodynamic time averages. In this case, entropy is a measure of microscopic freedom, essentially the volume of phase space available for system evolution: Given any initial state (and keeping system parameters fixed), the assumption is that the actual trajectory will get arbitrarily close to any point within this volume if you just waited long enough.

Why is Entropy's Definition Useful?

My somewhat facetious answer to that: Because it allows us to formulate the laws of thermodynamics.

Answer 2

What is entropy, more than disorder?

Don't look at entropy as disorder. Thinking of it as disorder has long been a source of confusion. Many texts are moving away from using the disorder description. Macroscopically, it's better to think of entropy as a measure of energy dispersion rather than as a measure of disorder. Microscopically, it's better to think of entropy as a measure of density of states rather than as a measure of disorder. The macroscopic and microscopic concepts come together in the field of statistical mechanics.

Why are there no units?

Entropy does have units. Entropy has units of energy/temperature, or Joules/Kelvin in the SI system. Entropy would essentially be unitless if temperature was just a measure of energy, but with goofy units. However, temperature is not energy. Temperature is related to but distinct from energy. A gram of ice at 0 °C and a gram of water at 0 °C have the same temperature but different energies.

Why is entropy useful?

Because it lets us explain why a couple of grams of liquid water at 5 °C doesn't turn into a gram of ice at 0 °C and a gram of water at ~90 °C. The first law of thermodynamics doesn't complain about this transformation. The second law most certainly does. Entropy puts limits on things and helps answer key questions such as "How much useful energy can this engine extract from a certain amount of fuel?"

Answer 3

Christoph gives a good answer from the point of view of statistical physics, but the first context we actually encounter entropy is in classical thermodynamics. It is useful even without the interpretation of being "the measure of disorder". This got slightly out of hand and is now quite technical, but I think it is still digestible and provides a complete explanation.

Recall the first law of thermodynamics, it tells us that the change of internal energy $\Delta U$ of a system is the energy expended by the system through external macroscopic parameters $-\sum W_i$ (i.e. sum of work done) and something "extra", which we usually call "heat" or $Q$ which just includes everything we cannot macroscopically grasp: $$\Delta U = \sum W_i + Q$$ So, to a certain extent this is a "law" as well as a definition which makes the law automatically apply. The non-trivial part of this is the fact that we say that the total energy is the only relevant internal parameter for a system in equilibrium.

So what about temperature? We see that certain systems transfer heat when in contact and some do not - the ones which do not, we say have same temperature and we postulate that it is a continuous well defined parameter $t$ of all thermodynamic systems. Since we also postulated that internal energy is the only relevant internal parameter of a system, there must be a one-to-one relation between temperature and internal energy $U=U(t), t = t(U)$.

When we pass to a differential formulation of the first law of thermodynamics, we get $${\rm d}U = - \sum a_i {\rm d} A_i + \delta Q$$ Where $A_i$ are all the macroscopic external parameters of the system such as volume and $a_i = -\partial U/\partial A_i$ are associated force-like parameters such as pressure.

We write only $\delta Q$ and not ${\rm d} Q$ because we know that when we go through a thermodynamic process, the expedition and gain of heat seems is dependent on the cycle we go through. That is, if we had $\delta Q = {\rm d}Q$ it would mean that expedition of heat in a cycle would be $$\oint \delta Q = \oint {\rm d}Q = 0$$ But this is not true. This fact of "cycle-dependence" is mathematically expressed as saying that $\delta Q$ is not a differential of a quantity, but a differential form. Differential forms can be expressed as $\delta Q = \sum f_i(A_i,t,U) {\rm d}X_i$, where $X_i$ can be again $A_i,t,U$. Since we have a one-to-one relation between $U$ and $t$, we can eliminate $U$ from these expression by substituting $U=U(t)$. Using a slightly more complicated physical argument we can show that $\delta Q$ can actually be expressed as $$\delta Q = f(t) {\rm d}S$$ where $S = S(t,A_i)$ is some function of the parameters of the system and $f(t)$ a monotonous function of $t$. Since $f(t)$ as a one-to-one function ("monotonous" in other words) can parametrize all systems very much like $t$, we can say it is a new temperature parametrization $f(t)=T$. That is, empirical temperature scales like Celsius and Fahrenheit would be $t$ and $T$ would be temperature in Kelvins. (The units don't matter, the scaling with respect to absolute zero does.)

This mysterious function $S(A_i,T)$ which lets us re-express $\delta Q = T {\rm d}S$ is the original definition of entropy. It is useful because it lets us trace the evolution of heat and thus of energy of a system in a very simple way. On the other hand, in classical thermodynamics, it is something of a place-holder for finding the full expression for ${\rm d}U$, since we have (using the chain rule) $$dS = \sum \frac{\partial S}{\partial A_i} {\rm d} A_i + \frac{\partial S}{\partial T} {\rm d} T$$ And we can thus trace changes of energy through variation of different measurable parameters such as $A_i$ and $T$. The units of physical entropy are actually $[Energy\times temperature^{-1}]=[k_B]$ where $k_B$ is the Boltzmann constant.

There is no need to invoke a disorder definition from the macroscopical point of view. You can actually think about entropy as a measure of possible heat transfer rather than disorder. A statistical physics analysis links entropy in the "thermodynamical limit" (number of particles $\to infty$ and volume $\to \infty$) to the dispersion of states through which the system is quickly switching. That is, the amount of disorder is found to be equal to one of the original very useful macroscopical quantities. But the fact is entropy was useful in the sense of the previous analysis even before formulating statistical physics.

Answer 4

As Christoph said, entropy is a measure of microscopic freedom. This is, I think, better than "measure of energy dispersal" (a now common description), for two reasons (and each one is enough for me):
1/ freedom can be measured in bits (or nats, digits, ...). 1 bit of freedom is the freedom to choose one of 2 options. entropy measures the freedom, the number of binary choices (if in bits) the system has to do to choose its state. What is 1 bit of energy dispersal is not clear.
2/ freedom is fundamental in nature, as shown in the EPR experiment, and also for me in the three-body problem chaotic nature.
Other reasons are:
Entropy is strongly connected to the degrees of freedom. It is then natural to say that it measures freedom.
Freedom is dynamic. One can see information in a deck of cards, but hardly freedom. Energy is required to manifest freedom. It's an help to avoid bad reasoning.
Inverse temperature is now seen in nat/joule. It is the factor which converts energy (in the form of heat) to freedom for the particles.
Shannon himself considered entropy as freedom of choice for the transmitter of the message. Here, the transmitter is the system, and the receiver the scientist (via a measurement).
Thermodynamic entropy is already measured in bits in some nanotechnology publications.
The relationship between thermodynamic freedom and freedom (in a philosophical sense) is the same as between information and meaning.
I like the idea that freedom is always increasing :-)

Answer 5

One thing entropy is NOT is a measure of energy dispersal. This erroneous idea is dangerously attractive because it is easy to understand and is sometimes true, but it leads to mistakes and should not be taken as a fundamental law.

A simple counterexample: Consider two boxes containing ideal gas, one at low temperature and one at high. The low temperature box has many more particles than the high temperature box, such that the total internal energy of the low temperature box is higher. Put the boxes in thermal contact, and which way will energy flow? Because of entropy and the second law, energy flows from high temperature to low temperature, which means it will flow from the low-energy box to the high energy one - that is, entropy will cause the concentration of energy, NOT its "dispersal".

One way to think of entropy that always applies is as a (logarithmic) measure of the number of independent microstates corresponding to a given macrostate. (It is often forgotten that entropy is defined with respect to a system's macrostate, and confusion often arises because people try to assign an entropy to a microstate by mistake. When asking, "What is the entropy of this system?" it is always useful to ask, "What are the macrovariables that define the system?") If there are more (microscopic) ways for a system to be in macrostate A than in macrostate B, then macrostate A has higher entropy. An ideal gas at temperature $T$ confined in volume $V$ has $k\ln 2$ less entropy than the same gas at the same temperature when its enclosure expands to volume $2V$, because there are twice as many places each molecule of gas could be.

Note that in that sense, entropy is also a measure of how much you don't know about the details of the state of a system (and by "you" I mean "the rest of the universe apart from the system"). If you know the exact location and momentum of every particle, there is only one way for the system to have that state, so its entropy is 0. "Entropy = ignorance."

Why is it useful? Because of usefulness! Originally thermodynamics was invented to try to develop a general theory about the efficiency of engines. Efficiency is the fraction of energy flowing through an engine that can be put to constructive use, leading to the need to distinguish "useful" and "unuseful" energy. Roughly, energy stored with low entropy is more "useful" than energy stored with higher entropy (at the same temperature).