6
$\begingroup$

From all sources I have seen it follows that the proof you can't decrease the amount of entropy in the Universe is given only statistically - the order is just one of the many ways how things can be (with the exception of only energy/temperature entropy, that's clear). That is my first question, is rule that entropy always increase valid for something else (than for entropy defined as an amount of balance of energy in the Universe? The only way out of that is, I think, define information as a physical quantity. Then we would know how much entropy increased.

Subset paradox

I have read this answer which defines information as a number of (minimum) YES/NO question you have to ask to perfectly specify the object that carries the information. But this would mean that everything (including every subset or superset which is impossible, how shows the picture) carries the same amount of information - for example if only describeable physical quantities were position and weight, my question for everything could be: "Is it true that it is there and it weighs that?"Now, let's consider a closed system consisting only of three particles.

Also following this definition of information it would be subjective what has more entropy - if I alphabetically order my books have I increased more entropy by the change in balance of energy in the room?

So how to define information correctly? (Btw this blew my mind - if the system had no spin, polarisation or local unbalance (the electrone has mole on one side) I wouldn't have any idea how to describe the position of them in the empty universe in other way than: It's here.)

$\endgroup$
  • 1
    $\begingroup$ You took the "Yes/No question" in the other answer a bit too literally. Have you never seen the definition of entropy/information as $\ln(\Omega)$ for $\Omega$ all possible microstates? $\endgroup$ – ACuriousMind Jul 12 '15 at 13:40
  • $\begingroup$ @ACuriousMind Yeah, that sounds good. $\endgroup$ – foggy Jul 12 '15 at 14:05
14
$\begingroup$

Entropy is subjective in the sense that you get to pick which macroscopic observables you care about keeping track of (usually, for instance, you care about things like temperature, pressure, etc.). Once you've defined the macroscopic observables, entropy is defined as the logarithm of the number of possible microstates that give rise to those macroscopic observables. If you define your "macroscopic" observables to be the position and momentum of every particle in your system (ignore, for the moment, the fact that quantum mechanics says that it is impossible to observe both at the same time), then the entropy is zero. If you include quantum mechanics, then there is inherent uncertainty about the trajectory through Hilbert space of you and the system you're interested in, so the entropy of any system you're interested in can never be zero regardless of how you define your "macroscopic" observables.

So to answer your question about rearranging your books: if the only macroscopic observable you care about is whether or not your books are alphabetized, then the entropy defined in terms of that macroscopic observable would decrease slightly when you arrange your books, although it would only decrease by an absurdly small amount because most of the entropy of your system of books comes from the microstates of the molecules in the books themselves, and not from the positions of the books. Furthermore, defining such an arbitrary macroscopic observable would really only be useful for your particular case and would not generalize to other systems of interest, which defeats the entire purpose of developing physical equations that can produce repeatable predictions in many different systems of interest.

$\endgroup$
  • 3
    $\begingroup$ Just for clarification, there is no uncertainty in the path through Hilbert space, just in the macroscopic observables that arise from an ensemble of quantum states. Quantum evolution is unitary. $\endgroup$ – Jerry Schirmer Jul 15 '15 at 11:35
  • 4
    $\begingroup$ Quantum evolution is only unitary for isolated systems. For any system large enough for entropy to be a useful consideration, the system will interact with the environment, so even though the evolution of [the system plus the environment] is unitary, there is uncertainty in the trajectory of [just the system] through its smaller Hilbert space because multiple copies of the system will arise as [the system plus the environment] moves through the larger Hilbert space. $\endgroup$ – Travis Jul 16 '15 at 1:34
3
$\begingroup$

You have a few confused ideas here- which is quite normal, because it is a confusing subject that is often not described very well. Instead of addressing them all specifically, I will answer the central question:

Entropy is objective.

It is true that one must specify exactly what degrees of freedom are in and out of your system to say what the entropy is. However, this is true for any other physical property, like energy, as well, so entropy is as objective as any other physical quantity.

Since I'm a physicist I'll talk about particles, but anything I'm saying can be adapted to cards or whatever system you're thinking of.

It's pretty obvious that the energy in a system depends on how you define the system. For example, adding particles into your system increases the total amount of energy you measure. Less obviously, you could choose to include or exclude certain degrees of freedom within each particle. For example, every particle with mass has associated energy given by $E=mc^2$, but if one doesn't know this or doesn't want to worry about it one could only consider the kinetic energy of the particles and ignore this degree of freedom.

Entropy is much the same*. Once you specify exactly which particles and which degrees of freedom within each particle you are keeping track of, it is uniquely specified. Choosing to ask more or less precise questions- such as asking exactly where a particle is in a bottle, versus just asking whether it is on the left or right side- amounts to ignoring more or fewer (or possibly zero) degrees of freedom.

There are two important differences between entropy and energy that I should note.

The first is that if you do include all the degrees of freedom of an isolated system, the entropy you calculate is always zero**. Since entropy is something like uncertainty, this is a statement that you have all of the possible information about the system. Once you start to leave parts out, the entropy becomes greater than zero.

The second is that, roughly speaking, in practice one can often ignore many degrees of freedom and still count the same total energy, while entropy is much more sensitive to this. This is because entropy is sensitive to correlations between particles, while energy is not. This is one way of describing the phenomenon of quantum entanglement.

See also my answer to a related, slightly more technical question: What is the entropy of a pure state?

*Technical qualification 1: I will have in mind the Von Neumann entropy, defined as $S=tr(\rho \log (\rho))$, with $\rho$ as the density matrix of the system. However, I believe that my statements apply to any other conventional definition of entropy in a physics context.

**Technical qualification 2: only if the system started in a pure quantum state.

Edit (07/15/15): Since this is clearly a contentious claim, I will give a toy example, at a higher technical level than my main answer. I will compare two example systems in which degrees of freedom are traced out.

First, an example of course graining in position. Let's say I have a particle that can be in one of four positions in a box:

__________________ | | | | A1 | B1 | | | | | A2 | B2 | |________|________|

There are sides A and B, and within each side are two sites.

The initial state is: $\rho=\frac{1}{\sqrt{2}}(|A1\rangle + |B2\rangle)\otimes \frac{1}{\sqrt{2}}(\langle A1| + \langle B2|) $

Or in matrix form: $\rho=\frac{1}{2}\left[ \begin{array}{cccc} 1&0&0&1 \\ 0&0&0&0 \\ 0&0&0&0 \\ 1&0&0&1 \end{array} \right]$

The columns and rows in this matrix are labeled like $|A1\rangle,|A2\rangle,|B1\rangle,|B2\rangle$. The entropy of this matrix, like any pure state, is 0.

Now, say we are only interested in whether the particle is on side A or B. We coarse grain by tracing out the extra degree of freedom:

$\rho_{red}=\langle 1|\rho|1\rangle+ \langle 2|\rho|2\rangle$ or $\rho_{red}=\frac{1}{2}\left[ \begin{array}{cc} 1&0 \\ 0&1 \end{array} \right]$.

The columns and rows in this matrix are labeled like $|A\rangle,|B\rangle$.

This is now a reduced density matrix for only the A/B degree of freedom. Since we started with an entangled state, the entropy is also now nonzero.

Now I'll turn to an apparently unrelated system, of two spin-1/2 atoms. The initial state is:

$\rho=\frac{1}{\sqrt{2}}(|\uparrow_1 \uparrow_2 \rangle + |\downarrow_1 \downarrow_2\rangle)\otimes \frac{1}{\sqrt{2}}(\langle \uparrow_1 \uparrow_2 | + \langle \downarrow_1 \downarrow_2|) $

Or in matrix form: $\rho=\frac{1}{2}\left[ \begin{array}{cccc} 1&0&0&1 \\ 0&0&0&0 \\ 0&0&0&0 \\ 1&0&0&1 \end{array} \right]$

The columns and rows in this matrix are labeled like $|\uparrow_1 \uparrow_2 \rangle,|\uparrow_1 \downarrow_2 \rangle,|\downarrow_1 \uparrow_2 \rangle,|\downarrow_1 \downarrow_2 \rangle$. We again start with a zero entropy pure state.

Now let's say we want to only consider one of the particles. We must trace out the degree of freedom associated with the other particle. But this goes through exactly in analogy to the previous example, and we end up with

$\rho_{red}=\frac{1}{2}\left[ \begin{array}{cc} 1&0 \\ 0&1 \end{array} \right]$.

The columns and rows in this matrix are labeled like $|\uparrow_1\rangle,|\downarrow_1\rangle$. At the level of the state structure of these two systems (which is all that determines entropy), the mapping between them is exact, and the entropy of the reduced system is again nonzero.

With all this setup, my point is simple. In my second example, I would be greatly surprised if anyone claimed that the entropy is arbitrary. You get two different results for the entropy by asking two different questions: what is the entropy of one particle versus what is the entropy of the full two-particle system. The fact that these have different answers is no deeper than the fact that you get different answers if you ask what the energy of one particle is versus the energy of two.

But exactly the same claim applies to the first system. In that case the degree of freedom you threw away didn't happen to be associated with a single particle, but it is still true that you would only get different answers for the entropy by asking different questions, and in particular by asking about different parts of the total system. There is no good reason to privilege degrees of freedom that happen to be contained in individual particles over those that do not. Since the only "subjective" aspect of entropy is the trivial fact that you have to choose precisely what you want to know the entropy of, it should be considered as objective as any other physical property.

$\endgroup$
2
$\begingroup$

I just want to amplify on what @Travis said and let me quote "Jaynes: Gibbs vs Boltzmann Entropies" who credits Wigner for this concept of THE "ANTHROPOMORPHIC" NATURE OF ENTROPY "...After the above insistence that any demonstration of the second law must involve the entropy as measured experimentally, it may come as a shock to realize that, nevertheless, thermodynamics knows of no such notion as the "entropy of a physical system." Thermodynamics does have the concept of the entropy of a thermodynamic system; but a given physical system corresponds to many different thermodynamic systems...... For example, the engineers have their "steam tables," which give measured values of the entropy of superheated steam at various temperatures and pressures. But the H20 molecule has a large electric dipole moment; and so the entropy of steam depends appreciably on the electric field strength present. It must always be understood implicitly (because it is never stated explicitly) that this extra thermodynamic degree of freedom was not tampered with during the experiments on which the steam tables are based which means, in this case, that the electric field was not inadvertently varied from one measurement to the next. Recognition that the "entropy of a physical system" is not meaningful without further qualifications is important in clarifying many questions concerning irreversibility and the second law. For example, I have been asked several times whether, in my opinion, a biological system, say a cat, which converts inanimate food into a highly organized structure and behavior, represents a violation of the second law. The answer I always give is that, until we specify the set of parameters which define the thermodynamic state of the cat, no definite question has been asked!"

$\endgroup$
  • $\begingroup$ I agree strongly with the content of this quote (especially the final sentence), but as a matter of terminology I don't find it useful to conclude that entropy is subjective. In Jaynes' example, it is certainly the case as well that changing the electric field would change the energy of the system of steam particles, and that any table of steam energy versus temperature would assume zero electric field, but would you say that it follows that energy is subjective? $\endgroup$ – Rococo Jul 13 '15 at 18:05
  • $\begingroup$ It would seem that internal energy is also subjective in a similar sense because $du=Tds + etc...$ depending on the "etc." but still I think I could meaningfully assign internal energy to a physical system without specifying the external control parameters. Yes, there is some arbitrariness of energy, its zero point but entropy has more arbitrariness. The changing electric field transfers energy to the system and with conservation what the outside has lost is what the inside has gained but entropy is not conserved unless the process is adiabatic and reversible. $\endgroup$ – hyportnex Jul 13 '15 at 21:07
  • $\begingroup$ Think of removing internal partitions in an isolated system leading to equalization of the inhomogeneities. That will not change conserved quantitites but surely will increase the entropy. $\endgroup$ – hyportnex Jul 13 '15 at 21:09
  • $\begingroup$ "Yes, there is some arbitrariness of energy, its zero point but entropy has more arbitrariness."- this statement itself seems rather arbitrary, no? ;) $\endgroup$ – Rococo Jul 14 '15 at 4:29
  • $\begingroup$ As mentioned in my answer, I agree that entropy is more sensitive to coarse-graining than energy. A minimal model for this is two spin-entangled particles (for definiteness, in a singlet state). If the particles are both considered individually, but their correlations are ignored, one will observe a nonzero entropy $S=2k_B ln(2)$ even though one still is keeping track of all the energy in the system. However, this entropy is unambiguously defined once one specifies the exact extent of the system (by giving a density matrix), in the same way as any other physical observable is. $\endgroup$ – Rococo Jul 14 '15 at 4:29
1
$\begingroup$

I only have a small comment to make on this topic. I think rather than use the word " subjective" it would be more exact to say that entropy is arbitrary, in the sense that, the value of entropy for your system depends on the variables you use to describe your macrostate, but once the choice is made, then the entropy is determined by the objective forces you are dealing with. Since entropy is based on the number of microstates of a system, which are in turn a reflection of our ignorance about a system, the value of entropy will change with how much we know(or choose to ignore) about a certain system. If two people agree to describe a system with the same set of variables, then they should also agree on the value of the entropy. If a third person were to describe system with a different set of variables, then the entropy will be different than the previous two people found. This is a statement not about the system itself, but rather the level of ignorance chosen between the two set ups. That being said, if you rearrange your book shelf, and it doesn't change the set of variables you are using to describe your books, no new information is gained or lost, so it shouldn't effect the entropy.

$\endgroup$
  • $\begingroup$ Okay, but do you think this is different than any other physical observable? I do not, for the reasons given in my edit to my answer. $\endgroup$ – Rococo Jul 15 '15 at 10:31
0
$\begingroup$

Von Neumann entropy is the quantity of information that is missing for a system since the rest of the Universe

Thermodynamic entropy is the quantity of information that is missing about a system becuase the macroscopical measurement devices only measure the macroscopic and the rest of thermodynamics quantities of the system (is the Von Neumann Entropy) of the measurement device, considering like measurement device system only characterized by the variables that are actually measurements of the thermodynamic observed system The last is subjective till you specify the concrete variables you are measuring

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.