# What is entropy really?

On this site, change in entropy is defined as the amount of energy dispersed divided by the absolute temperature. But I want to know: What is the definition of entropy? Here, entropy is defined as average heat capacity averaged over the specific temperature. But I couldn't understand that definition of entropy: $\Delta S$ = $S_\textrm{final} - S_\textrm{initial}$. What is entropy initially (is there any dispersal of energy initially)? Please give the definition of entropy and not its change.

To clarify, I'm interested in the definition of entropy in terms of temperature, not in terms of microstates, but would appreciate explanation from both perspectives.

-

## 11 Answers

There are two definitions of entropy, which physicists believe to be the same (modulo the dimensional Boltzman scaling constant) and a postulate of their sameness has so far yielded agreement between what is theoretically foretold and what is experimentally observed. There are theoretical grounds, namely most of the subject of statistical mechanics, for our believing them to be the same, but ultimately their sameness is an experimental observation.

1. (Boltzmann / Shannon): Given a thermodynamic system with a known macrostate, the entropy is the size of the document, in bits, you would need to write down to specify the system's full quantum state. Otherwise put, it is proportional to the logarithm of the number of full quantum states that could prevail and be consistent with the observed macrostate. Yet another version: it is the (negative) conditional Shannon entropy (information content) conditioned on the knowledge of the prevailing macrostate;

2. (Clausius / Carnot): Let a quantity $\delta Q$ of heat be input to a system at temperature $T$. Then the system's entropy change is $\frac{\delta Q}{T}$. This definition requires background, not the least what we mean by temperature; the well-definedness of entropy (i.e. that it is a function of state alone so that changes are independent of path between endpoint states) follows from the definition of temperature, which is made meaningful by the following steps in reasoning: (see my answer here for details). (1) Carnot's theorem shows that all reversible heat engines working between the same two hot and cold reservoirs must work at the same efficiency, for an assertion otherwise leads to a contradiction of the postulate that heat cannot flow spontaneously from the cold to the hot reservoir. (2) Given this universality of reversible engines, we have a way to compare reservoirs: we take a "standard reservoir" and call its temperature unity, by definition. If we have a hotter reservoir, such that a reversible heat engine operating between the two yields $T$ units if work for every 1 unit of heat it dumps to the standard reservoir, then we call its temperature $T$. If we have a colder reservoir and do the same (using the standard as the hot reservoir) and find that the engine yields $T$ units of work for every 1 dumped, we call its temperature $T^{-1}$. It follows from these definitions alone that the quantity $\frac{\delta Q}{T}$ is an exact differential because $\int_a^b \frac{d\,Q}{T}$ between positions $a$ and $b$ in phase space must be independent of path (otherwise one can violate the second law). So we have this new function of state "entropy" definied to increase by the exact differential $\mathrm{d} S = \delta Q / T$ when the a system reversibly absorbs heat $\delta Q$.

As stated at the outset, it is an experimental observation that these two definitions are the same; we do need a dimensional scaling constant to apply to the quantity in definition 2 to make the two match, because the quantity in definition 2 depends on what reservoir we take to be the "standard". This scaling constant is the Boltzmann constant $k$.

When people postulate that heat flows and allowable system evolutions are governed by probabilistic mechanisms and that a system's evolution is its maximum likelihood one, i.e. when one studies statistical mechanics, the equations of classical thermodynamics are reproduced with the right interpretation of statistical parameters in terms of thermodynamic state variables. For instance, a simple maximum likelihood argument, justified by the issues discussed in my post here one can demonstrate that an ensemble of particles with allowed energy states $E_i$ of degeneracy $g_i$ at equilibrium (maximum likelihood distribution) has the probability distribution $p_i = \mathcal{Z}^{-1}\, g_i\,\exp(-\beta\,E_i)$ where $\mathcal{Z} = \sum\limits_j g_j\,\exp(-\beta\,E_j)$, where $\beta$ is a Lagrange multiplier. The Shannon entropy of this distribution is then:

$$S = \frac{1}{\mathcal{Z}(\beta)}\,\sum\limits_i \left((\log\mathcal{Z}(\beta) + \beta\,E_i-\log g_i )\,g_i\,\exp(-\beta\,E_i)\right)\tag{1}$$

with heat energy per particle:

$$Q = \frac{1}{\mathcal{Z}(\beta)}\,\sum\limits_i \left(E_i\,g_i\,\exp(-\beta\,E_i)\right)\tag{2}$$

and:

$$\mathcal{Z}(\beta) = \sum\limits_j g_j\,\exp(-\beta\,E_j)$$

Now add a quantity of heat to the system so that the heat per particle rises by $\mathrm{d}Q$ and let the system settle to equilibrium again; from (2) and (3) solve for the change $\mathrm{d}\beta$ in $\beta$ needed to do this and substitute into (1) to find the entropy change arising from this heat addition. It is found that:

$$\mathrm{d} S = \beta\,\mathrm{d} Q\tag{4}$$

and so we match the two definitions of entropy if we postulate that the temperature is given by $T = \beta^{-1}$ (modulo the Boltzmann constant).

Lastly, it is good to note that there is still considerable room for ambiguity in definition 1 above aside from simple cases, e.g. an ensemble of quantum harmonic oscillators, where the quantum states are manifestly discrete and easy to calculate. Often we are forced to continuum approximations, and one then has freedom to define the coarse gaining size, i.e. the size of the discretizing volume in continuous phase space that distinguishes truly different microstates, or one must be content to deal with only relative entropies in truly continuous probability distribution models Therefore, in statistical mechanical analyses one looks for results that are weakly dependent on the exact coarse graining volume used.

-
Actually, entropy makes sense even outside of physics. We don't need a thermodynamic system to understand entropy. – DanielSank 18 hours ago
@DanielSank I used to be inclined to agree with this - I still agree with the second sentence, depending on one's definition of a "thermodynamic system". But in the light of the Landauer limit, even the information defined in abstract information theory must, for its realization, be encoded in the quantum state of some physical system and so entropy always, ultimately, refers to a physical system. – WetSavannaAnimal aka Rod Vance 12 hours ago
Mehhhh I kinda disagree. We can usefully talk about entropy in discussions about data encoding where the underlying physics is completely irrelevant. – DanielSank 11 hours ago
Relevant: physics.stackexchange.com/questions/263197/… (as elaborated in my answer there, I think DanielSank's formulation is more useful, but there is clearly significant disagreement about this). – Rococo 3 hours ago

Here's an intentionally more conceptual answer: Entropy is the smoothness of the energy distribution over some given region of space. To make that more precise, you must define the region, the type of energy (or mass-energy) considered sufficiently fluid within that region to be relevant, and the Fourier spectrum and phases of those energy types over that region.

Using relative ratios "factor out" much of this ugly messiness by focusing on differences in smoothness between two very similar regions, e.g. the same region at two points in time. This unfortunately also masks the complexity of what is really going on.

Still, smoothness remains the key defining feature of higher entropy in such comparisons. A field with a roaring campfire has lower entropy than a field with cold embers because with respect to thermal and infrared forms of energy, the live campfire creates a huge and very unsmooth peak in the middle of the field.

-
This answer seems to be regularly cited within SE, but I have never seen anything like it in the literature and it is not obvious to me how it would handle issues like entropy of mixing. – Rococo Sep 19 '15 at 16:40
Guilty as charged, I sometimes try to explain underlying concepts rather than just quote the equations or standard riffs. But I can assure you that in such cases the explanations I give are based on a very careful reading and analysis if those fundamental definitions; they are not just "analogies" or some form of loose interpretation. – Terry Bollinger Sep 20 '15 at 3:38
Regarding entropy of mixing, I'm genuinely surprised at your surprise! What could more represent a change towards smoothness at the molecular level of resolution than the intimate mixing of two or more previously macroscopically segregated populations of unique molecules? – Terry Bollinger Sep 20 '15 at 3:42
Smoothing of something, sure, but not of the energy distribution, which would not necessarily change during mixing. – Rococo Sep 20 '15 at 16:50
Also, if one looks at energy exchange between particles with different densities of states, it is easy to cook up a case in which the initial energy distribution is even and it becomes more uneven as the particles approach thermal equilibrium. So although this definition will roughly work in many cases, I think it is misleading. – Rococo Sep 20 '15 at 16:51

The entropy of a system is the amount of information needed to specify the exact physical state of a system given its incomplete macroscopic specification. So, if a system can be in $\Omega$ possible states with equal probability then the number of bits needed to specify in exactly which one of these $\Omega$ states the system really is in would be $\log_{2}(\Omega)$. In convential units we express the entropy as $S = k_B\log(\Omega)$

-

In terms of the temperature, the entropy can be defined as $$\Delta S=\int \frac{\mathrm dQ}{T}\tag{1}$$ which, as you note, is really a change of entropy and not the entropy itself. Thus, we can write (1) as $$S(x,T)-S(x,T_0)=\int\frac{\mathrm dQ(x,T)}{T}\tag{2}$$ But, we are free to set the zero-point of the entropy to anything we want (so as to make it convenient)1, thus we can use $$S(x,T_0)=0$$ to obtain $$S(x,T)=\int\frac{\mathrm dQ(x,T)}{T}\tag{3}$$ If we assume that the heat rise $\mathrm dQ$ is determined from the heat capacity, $C$, then (3) becomes $$S(x,T)=\int\frac{C(x,T')}{T'}~\mathrm dT'\tag{4}$$

1 This is due to the perfect ordering expected at $T=0$, that is, $S(T=0)=0$, as per the third law of thermodynamics.

-
Sir,then all initial entropy is 0 or not? – MAFIA36790 Aug 16 '14 at 2:14
@user36790: You can make it zero, but it doesn't mean it has to be zero. – Kyle Kanos Aug 16 '14 at 2:16
Sir,can u give me an example where initial $S$ is not 0? – MAFIA36790 Aug 16 '14 at 2:18
Not off the top of my head, but I would think there are some. – Kyle Kanos Aug 16 '14 at 2:19
@user36790 you cannot measure absolute value of energy, potential, nor entropy. In this cases, we usually define one simple case to be 0, but that is just convenience. Nothing changes if you decide your entropy at $T_0$ is 15400. – Davidmh Aug 16 '14 at 10:43

You can set the entropy of your system under zero temperature to zero in compliance with the statistical definition $S=k_B\ln\Omega$. Then the S under other temperature should be $S=\int_0^T{\frac{dQ}{T}}$.

-
Your second equation is more of a definition of the temperature in the microcanonical ensemble. – Jerry Schirmer Sep 20 '14 at 17:01

In classical thermodynamics only the change of entropy matters, $\Delta S = \displaystyle\int \frac{\mathrm dQ}{T}$. At what temperature it is put zero is arbitrary.

You have the similar situation with potential energy. One has to arbitrarily fix some point where the potential energy is put zero. This is because only differences of potential energy matters in mechanical calculations.

The concept of entropy is very abstract in thermodynamics. You have to accept the limitations of the theory you want to stick to.

By going to statistical mechanics one will get a less abstract picture of entropy in terms of the number of available states $\rho$ in some small energy interval, $S=k\ln (\rho)$. Still here we still have the arbitrary size of the small energy interval, $$S = k\ln (\rho) = k\ln\left(\frac{\partial \Omega}{\partial E}\Delta E\right)= k\ln\left(\frac{\partial \Omega}{\partial E}\right)+ k\ln(\Delta E)$$ Here $\Omega(E)$ is the number of quantum states of the system with energy lower than $E$. The last term is somewhat arbitrary.

-
So,sir,can I take initial entropy always 0 ? – MAFIA36790 Aug 18 '14 at 8:31
@user36790 Yes, as long as you are consistent in your choice of which state has S=0, you can do as you like. – Per Arve Aug 21 '14 at 18:11

The definition of a physical concept can be a differential form but can’t be the difference of functions. $\Delta S=S_{final}-S_{initial}$ is an equation but not the definition of entropy. Thermodynamics itself now can hardly explain “what is the entropy really" , the reason please see bellow.

1.Clausius’ definition

\begin{align}\mathrm dS=\left(\frac{\delta Q}{T}\right)_\textrm{rev}\end{align}

Questions: 1) Since $\displaystyle \oint \delta Q/T\le 0$, $S$ cannot be proved to be a state function in maths, it can only depend on the reversible cycle of heat engine, this does not seem like a perfect foundation in the usual sense, and is an only exception as the definition of the state function both in mathematics and physics. As a fundamental principle, the state function changes must be independent of the path taken, why the definition of the entropy is an exception? 2) Clausius’ definition cannot explain the physical meaning of the entropy.

1. The fundamental equation of thermodynamics

\begin{align}\mathrm dS=\frac{\mathrm dU}{T}-\frac{Y~\mathrm dx}{T}-\sum_j\frac{\mu_j~\mathrm dN_j}{T}+\frac{p~\mathrm dV}{T}.\end{align}

Questions: 1) The equation includes the difference of functions, what is this difference? 2) The equation cannot explain the physical meaning of the entropy.

3) Boltzmann entropy

\begin{align}S=k\ln\Omega. \end{align}

Question: 1) $\Omega$ depend on the postulate of the equal a priori probability, but this postulate does not need to be considered in thermodynamics. In general, the postulate of the equal a priori probability cannot hold for mechanics potential energy and Gibbs free energy, a chemical reaction comes from the gradient in chemical potentials $\Delta \mu$ but not the equal a priori probability. The postulate can be applied to describe thermal motion but is not suitable for interactions.

-
+1 for bringing up the fundamental thermodynamic relation (in its most general form), which is really the most precise way to define changes in entropy in terms of changes in other thermodynamic variables, if one doesn't want to get into the statistical mechanics meaning of entropy. – Hypnosifl Jan 5 '15 at 15:51

First, you have to understand that Rudolf Clausius put together his ideas on entropy in order to account for the losses of energy that was apparent in the practical application of the steam engine. At the time he had no real ability to explain or calculate entropy other than to show how it changed. This is why we are stuck with a lot of theory where we look at deltas, calculus was the only mathematical machinery to develop the theory.

Ludwig Boltzmann was the first to really give entropy a firm foundation beyond simple deltas through the development of statistical mechanics. Essentially he was the first to really understand the concept of a microstate which was a vector in a multidimensional space (e.g. one with potentially infinite dimensions) that encoded all of the position and momentum information of the underlying composite particles. Since the actual information about those particles was unknown, the actual microstate could be one of many potential vectors. Entropy is simply an estimate of the number of possible vectors that actually could encode the information on the particle positions and momentums (remember, each individual vector on it own encodes the information about all the particles). In this sense entropy is a measure of our ignorance (or lack of useful information).

It is this latter use of entropy to measure our level of knowledge that led Claude Shannon to use the machinery of entropy in statistical mechanics to develop information theory. In that framework, entropy is a measure of the possible permutations and combinations a string of letters could take. Understanding information entropy is very critical to understanding the efficacy of various encryption schemes.

As far as defining Temperature in terms of entropy. These are general viewed as being distinct but related measures of the macrostate of a system. Temperature- entropy diagrams are used to understand heat transfer of a system. In statistical mechanics the partition function is used to encode the relationship of temperature and entropy.

http://farside.ph.utexas.edu/teaching/sm1/lectures/node64.html See eq 420, temp is embedded in definition of beta

http://en.m.wikipedia.org/wiki/Rudolf_Clausius#Entropy

http://en.m.wikipedia.org/wiki/Claude_Shannon

http://en.m.wikipedia.org/wiki/History_of_entropy

http://en.m.wikipedia.org/wiki/Ludwig_Boltzmann

http://en.m.wikipedia.org/wiki/Temperature–entropy_diagram

P.s. Apologies I would normally embed my links but am mobile at the moment

-

As a general rule, physics gets easier when the mathematics gets harder. For example, algebra-based physics comprises a bunch of seemingly unrelated formulae, each and every one of which needs to be memorized separately. Add calculus and wow! Many of those supposedly disparate topics collapse into one. Add mathematics beyond the introductory calculus level and the physics gets even easier. The Lagrangian and Hamiltonian reformulations of Newtonian mechanics are much easier to grasp -- so long as you can understand the mathematics, that is.

The same applies to thermodynamics, in spades. There used to be a website that provided 100+ statements of the laws of thermodynamics, the vast majority of which addressed the second and third laws of thermodynamics. The various qualitative descriptions were quite hair-pulling. Most of those hair-pulling difficulties vanish when you use the more advanced mathematics of statistical mechanics as opposed to the sophomore-level mathematics of thermodynamics.

For example, consider two objects at two different temperatures in contact with one another. The laws of thermodynamics dictate that the two objects will move toward a common temperature. But why? From the perspective of thermodynamics, it's "because I said so!" From the perspective of statistical mechanics, it's because that common temperature is the one temperature that maximizes the number of available states.

-
This is a correct and helpful observation and something that many posters on this site should probably hear. But ... it doesn't really address the question; instead it explains why you are not going to answer the question. – dmckee Aug 16 '14 at 17:05

A higher entropy equilibrium state can be reached from the lower entropy state by an irreversible but purely adiabatic process. The reverse is not true, a lower entropy state can never be reached adiabatically from a higher entropy state. On a purely phenomenological level the entropy difference between two equilibrium states, therefore, tells you how "far" away they are from being reachable the lower entropy state from the higher entropy one by purely adiabatic means. Just as temperature is a scale describing the possibility of heat flow between interacting different temperature bodies, entropy is a scale describing the states of a body as to how close or far apart those states are in the sense of an adiabatic process.

-

What is entropy really?

I want to answer(!) this question from a different point of view.

First off, I focus on your title and the phrase “really”. We don’t know what entropy is really. We don’t know what energy is really also, and any thing or concept else too. Entropy, like all other concepts created by humans, is a convention between some people to refer to the same(!) thought or sense.

I want to mention an example here. I want to ask “What is green color really?” The answer is same: “We don’t know.” But we usually talk about that with other people and if no one knows what green is, so how they can understand their aims? How has this concordance (or synchrony (sorry because of poor English)) between humans’ thoughts or senses been created? The answer is “Passing of time creates that concordance”. Passing of time helps us to understand each other without knowing that how we understand each other!

I think another intuitive example can help. I want to refer to ordinary education of children. No child knows why $1+1=2$ or $1$ plus $1$ is equal to $2$ (honestly, I don’t know also, even now!). They just see(!) some shapes like $1$, $1$, $+$, $=$ and $2$, and the cleverest ones those can analyze more than their classmates, say with themselves “When I see theses shapes $1+1=$, I must draw this shape $2$ after the $=$.” In fact, they don’t think about “What those shapes are” and what is important and wonderful here is that after some period (passing of time) and repetition they think that they have learnt addition and it was so simple! This process is occurring at high levels and ages too. One of my professors was saying (I don’t know it was true or not, I just quote): “If you ask from a Japanese engineer what stress is, he/she cannot answer you but they build nice bridges, machines, etc.” I think if they cannot talk about reality of stress, that is because they have passed the same process of education. When engineering students see the formula of stress $\sigma=\frac FA$, it is strange for them. When they write that formula themselves, its strangeness reduces a little bit. And after some period and repetition, they think that stress never has been strange for them while they don’t know what stress is even after passing of time and repetition.

Maybe you say that you have seen some people that are able to talk about stress for hours. You are right but even those people don’t know what stress is. Because for defining stress, they get help from other concepts or things and as I mentioned before, we don’t know all concepts and things. Their explanation is too much useful for decreasing the time that is necessary for creating the common thought or sense, but doesn’t remove the ignorance.

So, if you cannot understand the entropy, I should say: “No worry. You will understand it without that you know how you have understood it if you are patient and repeat. Passing of time will do her job well!”

If you want a physical explanation about entropy, I think you have already been answered by other users and I have seen other nice answers of your question in other posts also (you can see them by searching PSE posts).

-

## protected by Qmechanic♦Jan 5 '15 at 20:25

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?