I have problem understanding entropy because of some contrary examples

Question

In many places over the Internet, I have tried to understand entropy.

Many definitions are presented, among which I can formulate three (please correct me if any definition is wrong):

1. Entropy = disorder, and systems tend to the most possible disorder
2. Entropy = energy distribution, and systems tend to the most possible energy distribution
3. Entropy = information needed to describe the system, and systems tend to be described in less lines
4. Entropy = statistical mode, and the system tends to go to a microscopic state that is one of the most abundant possible states it can possess.

Now, I have these contrary examples in my mind:

1. Disorder => how about a snowflake? What is disorder? How do we agree on what is ordered and what is disordered? Because to me a snowflake is a perfect example of order.
2. Energy distribution => then why Big Bang happened at all? As they say, universe was one tiny point of energy equally distributed. Now the universe is a repetition of energy density and void.
3. Information => we can describe the universe before the Big Bang in one simple sentence: an energy point, X degrees kelvin. But we need billions of billions of lines of descriptions to be able to describe the universe.
4. Mode => Again, before the big bang, or even in early epochs of the universe we had uniform states that were the most abundant possible states.

I'm stuck at this very fundamental philosophical definition.

I can understand the "cup of coffee" example of course, or the "your room gets messy over time" example. Those are very clear examples. But I'm stuck at these examples. Can you clarify this for me please?

Answer 1

Your concern about the too many definitions of entropy is well-founded. Unfortunately, there is an embarrassing confusion, even in the scientific literature on such an issue. The answers you may find even in the SE sites just mirror this state of things.

The short answer is that there is nothing like a unique concept of entropy. There are many different but correlated concepts, which could have been named differently. They have some direct or indirect relation with thermodynamic entropy, although they usually do not coincide with it without additional assumptions.

Just a partial list of different concepts, all named entropy contains

1. Thermodynamic entropy.
2. Dynamical system entropy.
3. Statistical mechanics entropy.
4. Information theory entropy.
5. Algorithmic entropy.
6. Quantum mechanics (von Neumann) entropy.
7. Gravitational (and Black Holes) entropy.

Although all these quantities are named entropy, they are not entirely equivalent. A schematic list of the range of systems they can be applied and some mutual relation could help organize a mental map in such a confusing conceptual landscape.

Let me add a preliminary disclaimer. I am not going to write a comprehensive treatise on each possible entropy. The list is intended as an approximate map. However, even if I may be missing some important relation (I do not claim to be an expert on every form of entropy!), the overall picture should be correct. It should give an idea about the generic non-equivalence between different entropies.

1. Thermodynamic entropy

It can be applied to macroscopic systems at thermodynamic equilibrium or even non-equilibrium systems, provided some sort of a local thermodynamic equilibrium (LTE) can be justified for small regions of the system. LTE requires that each subregion is large enough to neglect the effect of relative fluctuations (local thermodynamic quantities have to be well defined), and the relaxation times are faster than typical dynamic evolution times. Usual thermodynamics requires the possibility of controlling the work and heat exchanged by the system and crucially depends on some underlying microscopic dynamics able to drive the system towards equilibrium.

2. Dynamical system entropy

The present and other items should contain sublists. Under this name, one can find entropies for abstract dynamical systems (for example, the metric entropy introduced by Kolmogorov and Sinai) and continuous chaotic dynamical systems. Here, the corresponding entropy does not require an equilibrium state, and recent proposals for non-equilibrium entropies ( an example is here ) can be classified under this title.

3. Statistical mechanics entropies

Initially, they were introduced in each statistical mechanics ensemble to provide a connection to the thermodynamic concept. In principle, there is one different entropy for each other ensemble. Such different expressions coincide for a broad class of Hamiltonians only at the so-called thermodynamic limit (TL), i.e., for systems with a macroscopically large number of degrees of freedom. Notice that Hamiltonians have to satisfy some conditions for TL could exist. Apart from the coincidence of entropies in different ensembles, TL is also required to ensure that the statistical mechanics' entropies would satisfy some key properties of thermodynamic entropy, like convexity properties or extensiveness. Therefore, one could say that the statistical mechanics' entropy is a generalization of the thermodynamic entropy, more than being equivalent.

3. Information theory entropy

This entropy is the well-known Shannon's formula $$ S_{info}= -\sum_i p_i \log p_i $$ where $p_i$ are the probabilities of a complete set of events.

It is clear that $S_{info}$ requires only a probabilistic description of the system. There is no requirement of any thermodynamic equilibrium, the energy of a state, and no connection exists with work and heat, in general. $S_{info}$ could be considered a generalization of the statistical mechanic entropy, coinciding with that only in the case of an equilibrium probability distribution function of thermodynamic variables. However, $S_{info}$ can be defined even for systems without any intrinsic dynamics.

4. Algorithmic entropy

In the present list, it is the only entropy that can be assigned to an individual (microscopic) configuration. Its definition does not require large systems, probability distribution, intrinsic dynamics, or equilibrium. It is a measure of the complexity of a configuration, expressed by the length of its shortest description.

The relation of algorithmic entropy and information entropy is that if there is an ensemble of configurations, the average value (on the ensemble) of the algorithmic entropy provides a good estimate of the information entropy. However, one has to take into account that the algorithmic entropy is a non-computable function.

6. Quantum mechanics (von Neumann) entropy

Although different from the formal point of view can be considered a generalization of Shannon's ideas to describe a quantum system. However, concepts like thermal equilibrium or heat do not play any role in this case.

7. Gravitational (and Black Holes) entropies

A set of stars in a galaxy can be thought of as systems, at least in LTE. However, their thermodynamics is quite peculiar. First of all, it is not extensive (energy grows faster than the volume). The ensembles' equivalence does not hold, and it is well known that the microcanonical specific heat is negative. A similar but not precisely equal behavior is found for the Black Hole entropy proposed by Beckenstein. In this case, the quantity that plays the role of entropy is the area of the horizon of the events of the Black Hole. Although it has been shown that this entropy shares many properties of thermodynamic entropy and can be evaluated within String Theory by counting the degeneracy of suitable states, its connection with thermodynamic entropy remains to be established

What about disorder?

It remains to discuss the relation between entropies (plural) and disorder.

It is possible to associate to each entropy a specific concept of disorder. But it is easy to guess that, in general, it won't be the same for all.

The only disorder associated with thermodynamic entropy is the disorder connected to how extensive quantities are stored in different subsystems of the same macroscopic state. Within thermodynamics, a well ordered macroscopic state is a state where extensive quantities are spatially concentrated. The maximum disorder coincides with a spread of the extensive state variables to ensure the same temperature, pressure, and chemical potential in each subvolume.

Within classical statistical mechanics, one can associate disorder to the number of available microstates in the phase space. Notice however, that this disorder, in general, has nothing to do with the usual definition of spatial order. The reason is connected with the non-intuitive role of inter-particle interactions and the fact that the statistical mechanic entropy is related to counting the number of microstates.

Probably, the entropy with the closest connection with the usual meaning of disorder is the algorithmic entropy. But that is also the most difficult to evaluate and the farthest from the thermodynamic entropy.

---

A small postscript

A pedagogical illustration of the complete decoupling between configurational order and entropy comes from the Sackur-Tetrode formula for classical ideal gas entropy. It shows that the entropy is directly proportional to the atoms' mass, while the accessible configuration space and the probability of each spatial configuration are the same.

Answer 2

The reason why Entropy has so many descriptions is not because it was designed to. Nobody started out with all of those things called Entropy.

Entropy started off with one thing. And then a bunch of other stuff was found to be related, both mathematically and physically, to that one thing.

Way back, it was observed that all useful energy ends up turning into useless and diffuse heat. This process was called Entropy. They knew it happened. They didn't know why.

So that is the start. Entropy, when originally named, was just a description of something that happens whenever you poke at the universe and look at what happens. That Energy becomes useless waste heat.

Every other definition of Entropy is because either it was a way to describe why or how that happens, or because the mathematics lined up with other Entropy mathematics. And surprisingly often that lining up of mathematics actually ends up having a physical meaning.

This is known as the unreasonable effectiveness of mathematics in the natural sciences; mathematical patterns keep on explaining things about the universe, and naively this is very surprising and unreasonable.

---

So, going back to Entropy. We start with observational science. That waste heat thing:

It was observed that heat flows from hot things to cold things. This was mathematically modeled with values like temperature and heat flow. This is a law of Entropy, that heat energy flows from hot things to cold things and not other way around. From it you can generate an insane amount of descriptive power about the universe.

Along comes Boltzman, who takes that "heat flow" and describes it more abstractly. He describes "macrostates"; something we can describe at our chosen scale. You divide your description of what could happen into a bunch of macrostates (as many or as few as you want).

In each macrostate there are many, many indistinguishable (at our scale) "microstates" that produce the "same" macrostate.

A microstate is a microstate because, while it is different than other microstates, it is in a way that we don't care about when we originally described our macrostates.

For example, "there is a table in the room" is a macrostate. The scratches on the table, the exact location of the termite in it, the current velocity of the atom of carbon in the geometric center of the table -- all of those are not described by my macrostate. So, we call all of the actual, physical states we clumped into that macrostate to be microstates.

If you count those microstates in each macrostate, you find that a closed system almost always moves into the more common "macrostates" from the rarer ones. And this is sufficient to describe the transfer of energy from hot objects to cold ones; the number of microstates in two tepid objects is insanely higher than the number of microstates that describe one hot and one cold object.

This is strange. But, as it turns out, that when we actually go and count how many states a given macrostate has, instead of getting "table in room has 10 billion states" and "rubble of table has 15 billion states" -- ie, the two numbers are relatively similar, we get something crazy like "rubble case has $10^{1000000}$ times as many microstatesthan table does". (the exact number is not accurate, the point is that it is a ridiculously huge factor, not a small one)

This is so true that we end up measuring the number of microstates by taking the logarithm. So we get the table has X Entropy, and the rubble has X+1000000 Entropy. Only a million more units of entropy; but because this is on an exponential scale, that is actually $10^{1000000}$ times more states.

This statistical description of Entropy matches the earlier one -- it explains why heat energy flows from hot to cold objects, and why useful energy ends up being emitted as "useless" diffuse waste heat.

Weird. But not weird enough. Now things get strange.

Way off in mathematics, someone was working on a subject called Information Theory. This is useful to do things like figure out how to pass more information along a wire or radio signal; how much can you send? Can you improve on this protocol with one that sends more? How do you fix errors caused by random noise? Given an English sentence or piece of music or a picture, how much can you compress it and still get the original back afterwards?

Shannon generated a measure of information in a system. And, somewhat amazingly, it ends up working like physical Entropy does; the same mathematical equations govern both of these. And, with work, you can connect Shannon information entropy to Boltzman statistical entropy in physical ways.

From there you get further abstractions and remixing. Things that "behave like" Entropy in a new domain are called Entropy. And often when connected back to macroscopic physics and the transfer of heat it is the same phenomena and deduces "energy tends to become useless, diffuse heat".

---

Now, part of your confusion is that you are looking back to the big bang, and saying "but that was a state of really low Entropy!".

And yes, it was. We are going to have a far lower entropy than the universe when Big Bang happened has.

Why did the Big Bang happen? That isn't explained by Entropy. Entropy tells us why the Big Bang leads to us, and why the Big Bang must be a point of extremely low Entropy. Not every piece of reality is explained by every piece of scientific theory. In order to investigate the Big Bang's "origins" you'll have to use more than the laws of Entropy.

Entropy applies to a closed system; parts of that system can have reduced Entropy, but only at the cost of increasing the Entropy elsewhere in the system more. Snowflakes or Humans are not contradictions of the laws of Entropy, because in both cases they formed as part of a larger system.

Also, your information description is backwards. Entropy is a measure of how much information it would require to fully describe a system, and it never decreases. This means that the Big Bang, as a low entropy state, is the simplest phase of the universe to fully describe.

Now, this "full description" tends to be extremely boring. You are doing something like describing the location and movement of every single particle, individually (I am ignoring QM here; it has its own definition of entropy that is consistent, but not something I'm going into here). When you have a room full of gas bouncing around at random, that is harder to describe than the same number of particles all arranged in a regular grid.

Suppose we take the room full of gas, freeze it, and carve a fancy sculpture out of the resulting crystal. To us, the shape that crystal takes is more interesting than the boring "room full of gas". But fully describing that crystal statue turns out to be insanely easier; the particles are more constrained in position and velocity, they form a regular grid instead of a chaotic gas. The exact shape of the crystal doesn't require all that much information, but constrains the number of states the atoms can be in a huge amount. It is a very low entropy state, when you fully describe everything.

We are just bored by the room of gas, but interested in the crystal statue, so we talk more about the statue than the room of gas.

Humans often like low-entropy things. Our brains are pattern-matchers, and low-entropy things have lots of patterns. High entropy things tend to be "boring" smears, as constraining things to a pattern is a massive reduction in the positions the atomic scale particles can be in.

---

Let's get concrete.

So why can't we unbreak an egg?

The "macro/macrostate" trick is a bit fun. Counter intuitively to us, the high entropy macrostates have insanely large numbers of microstates compared to the low entropy states. When we convert entropy to a number, we take the logarithm of the number of microstates. So every 'unit' of entropy is an exponential increase in the number of microstates.

A 'high' entropy situation could have an entropy thousands or millions of units greater; now take e and raise it to a power of a million. That is how many more times greater the number of microstates the high entropy macrostate has.

If going from one state to another is anywhere close to uniform, going from a macrostate with X times $10^{1000000}$ more states back to a state with X states is going to be, well, not very likely. And that is what happens when you want to unbreak an egg. There are a simply ridiculous number of "broken egg" states, and very very few "unbroken egg" states. The dropping of the egg on the floor disrupts the somewhat stable "unbroken egg" macrostate, and moves it into a random state in the (broken egg + unbroken egg) combined state.

Going from the (unbroken egg + broken egg) combined state back to an unbroken egg requires that we reach one of those X states among the X times $10^{1000000}$ combined broken and unbroken states.

So now you feed the broken egg to a chicken (well, many broken eggs). And out comes a single unbroken egg. How?

The chicken takes the still-low-entropy molecules in the broken egg, and uses their "ordered" energy to order other molecules inside itself. This engine emits heat -- high entropy energy -- and concentrates some low entropy materials inside the chicken. Those low entropy materials are in turn converted into high entropy waste, and used to do build other low entropy materials (new cells, create membranes that concentrate calcium, make blood carrying sugars and oxygen go to a cell that will grow into an egg, transcribe DNA into RNA and RNA into proteins, etc).

After consuming a pile of low-entropy matter and converting it into higher-entropy heat and waste matter (poop!), it takes some matter and arranges it into an egg.

This process is not 100% efficient. That chicken produced more entropy in waste products than the difference between the raw materials and the finished egg has. A closed system with a chicken, which lays an egg, you then feed the egg back to the chicken, cannot produce new eggs without the chicken's biological engine damaging itself.

Typically the input to this process is via feeding the chicken plant materials, which in turn turned CO2 in the air into low entropy plant matter by absorbing low-entropy light from the sun.

The sun in turn produces light by taking low entropy hydrogen and fusing it into higher entropy helium. The pressure to do that was fueled by gravitational collapse, where a non-uniformity in interstellar gas caused some to clump, radiate heat as it fell in (that heat being high-entropy energy), pull in more gas, and grow until the center was hot and pressurized enough to start fusion.

The hydrogen fuel for the sun was left over from the low entropy big bang. Early in the big bang it was too hot for neutrons and protons to stay stuck together. As it cooled, they started fusing, but the rate of cooling was so fast that not all of the Hydrogen became Helium, and there wasn't enough time at the required pressure and temperature to fuse everything into Iron (the highest entropy atomic nucleus arrangement of neutrons and protons).

---

Imagine the world as an extremely steep slope that is also extremely, extremely long.

Bouncing down the slope are boulders. These boulders bounce off the ground, losing energy. As they do so they lose forward momentum.

But they are on a slope, so they also fall. This keeps them going.

Trying to get a boulder to roll uphill in the middle of this avalanche is insanely hard. Getting it to roll downhill is very easy.

Now, you could even use the boulder's to build a pattern, but that pattern has to roll down the hill as well; it can't stay stationary. The slope is too steep.

The universe, as best we can tell, is an extremely steep entropy slope from the big bang. We harvest leftover low entropy matter -- mostly hydrogen -- from the big bang, convert it to low entropy light, convert that to carbon plants, bury all of it and have it decay into hydrocarbon, burn those hydrocarbons to run our coal plants, make that vibrate electrons to produce electrical power, use that to convert aluminum ore into pure metal and run machines that stamp out cans of it, then open the can and drink some clean water from it.

Each of those harvests is like using the energy from one of those falling boulders (as we ourselves are also falling) in order to get things done.

Answer 3

The general expression for the entropy of a system that is in a particular macrostate in terms of $\Omega$, the number of microstates associated with that macrostate. $$S=k_B\ln\Omega$$ We are assuming that the system is in a particular macrostate with fixed energy.

---

The above will work for all the cases!

You should ask different questions for all the doubts. The following might be useful

• What Was The Entropy Of The Universe At The Big Bang?
• What entropy really is?
• Intuitive explanation of entropy
• Why does the formation of snowflake does not violate the second law of thermodynamics?

Answer 4

A snowflake is indeed quite orderly. There are fewer ways in which a bunch of water molecules can make up a snowflake than there are ways in which they can make up a droplet of water, with all of them moving all over the place.

But that only means that the snowflake forms in a process that causes entropy to increase somewhere else. Water that freezes gives off heat - that heat increases the entropy of its surroundings. The same is true for living organisms: they are very orderly matter, but at the price of eating, and so disrupting, larger amounts of matter.

Answer 5

tl;dr– Entropy's ignorance. That's it. The other descriptions are approximations or/and special cases. This partial-answer points out that entropy is subjective, rather than context-independent, to help address a common point-of-confusion.

---

Entropy isn't real.

Entropy is model-subjective, including context/observer-subjective. It's not a real, universal value; it doesn't exist independent of context. Understanding this will help avoid a lot of confusion about apparent contradictions.

Thought experiment that's commonly misinterpreted.

Thought experiment:

1. Consider an ideal gas in a glass jar. At an initial time, one side has helium, the other has neon.

2. As time starts, we tend to expect the gases to mix into a fairly even mixture throughout, which is often described as a high-entropy state.

3. Sometimes, the gas will spontaneously de-mix, with helium and neon again separated, e.g. as at initial time.

4. Did entropy return to its initial value?

Here's the thing: entropy isn't real. It belongs to the model, not the actual physical system. So what an actual physical system does doesn't matter because entropy was never about that actual physical system in the first place.

Rather, what entropy would tell us is that, after arbitrarily much time, the system can be predicted to be fairly selected from the ensemble of possible arrive-able states, the vast majority of which aren't de-mixed. This isn't violated by a system de-mixing as this predicts de-mixing as a possible (if rare) state.

If an experimenter observes a system that they've considered to be at max-entropy, and finds it de-mixed, then they can again consider that physical system at low-entropy. This, again, isn't a contradiction: they've merely constructed a new model, informed by physical observation. The old model which predicts max-entropy at the same time is still, likewise, valid. These models may exist simultaneously without contradiction (though, obviously, we'll tend to much prefer a more-informed model).

Conclusion: Entropy is subjective, not real.

Entropy's a huge topic that's involved in a lot of things. It's ultimately a qualification of ignorance, which can be formally defined, though it gets wrapped up with a lot of case-specific mechanics that cause the word "entropy" to imply additional things in different contexts.

The suggestion of this answer is to keep a focus on its subjectivity: that it's always about the observer's context-dependent model, not real-world systems. Don't forget that the de-mixed gas can be considered either max-entropy or min-entropy, depending on the observer's contextual frame, without contradiction.

---

Related: Probability isn't real.

Probabilities also aren't real, which might be a better starting-point than entropy (since probability's a bit simpler).

A pretty funny question was

• "How to explain that winning the lottery is not a 50/50 distribution?",

in which the OP's child thought that winning the lottery was 50% likely. The funny thing is that, however odd that may've seemed, the kid was right! Not that I'd recommend anyone buy lottery tickets, but rather that the kid correctly applied statistical reason in the normal fashion, arriving at a defensible (if humorous) result.

The critique of the "50%-odds"-assessment wouldn't be that it's wrong, in a technical sense, but rather ill-informed. It's like the "de-mixed-ideal-gas-is-at-max-entropy"-assessment: it's correct, just ill-informed of the observation that the gas de-mixed.

In short, I'd suggest thinking about how probabilities work and why they're inherently subjective. This can connect well with figuring out entropy.

---

Thought experiment: The Monty Hall problem.

The Monty Hall problem is a funny pop-culture thought-experiment that also demonstrates subjective probability.

Basically:

1. There's a game show with 3 doors, with a prize behind one.

2. A contestant gets to pick a door, then the host removes one of the other doors which didn't have the prize behind it.

3. The contestant is allowed to switch from their current selection to the other, remaining door. Should they?

Apparently some folks don't switch. They think that probability is real (context-independent), and since the remaining door was as likely as their initial selection, the odds wouldn't be improved by switching.

Of course, they're missing an important observation: the two other doors, together, had 2-in-3 odds of being correct, and since a wrong one was removed, the contestant can get that 2-in-3 chance by selecting the remaining door. By contrast, their current door still has only 1-in-3 odds.

The lesson to focus on here is that probability/entropy/etc. don't belong to the actual physical system, but rather to models of it.

Answer 6

I think the sheer number of answers to this question tells you that this is not an easy question. While it may be folly to add (yet another) answer to the pile, I wanted to give (yet another) slightly different perspective.

I think the problem is not that entropy is ill-defined or ambiguously defined. Entropy has a clear and precise definition. Given a probability density function $p(x)$ that depends on some variables $x$, the entropy is \begin{equation} S \equiv -\int {\rm d} x p(x) \log p(x) \end{equation}

However the typical way we calculate and use entropy requires us to make further choices, and one will make different choices in different applications.

• First, in order to calculate the entropy for a given distribution (in other words to do the integral above), we need to specify the space of possible states (we need to know all possible values of $x$), and we need a measure ${\rm d} x$ on this space. In order to define this space, we may need to consider constraints, to determine if a given state is or is not considered "possible."

• Second, we usually aren't interested in the entropy per se. We are interested in finding the probability density function that maximizes the entropy. In order to maximize the entropy we need to specify what is the space of probability density functions over which we are going to maximize.

In physics (specifically in thermodynamics / statistical mechanics), we typically are interested in probability distributions over the space of microstates of a system. For instance, for a gas, we would define a probability distribution over every possible configuration of the gas -- every possible position and momentum for each gas molecule.

As mentioned above, we have to consider the constraints placed on the system. We can consider that the system has a fixed total energy and number of particles. Then the probability distributions will only be functions of microstates that have the prescribed energy and particle number. The resulting probability distribution that maximizes the entropy under this constraint is called the micro-canonical ensemble. We can also consider that the energy of the system can vary but the number of particles is fixed (the canonical ensemble), or that the energy and number of particles can vary (the grand-canonical ensemble). Depending on the space of allowed microstates, the probability distribution that maximizes the entropy will be different. This is not surprising -- as you change the space of "allowed" distributions it is not surprising that you will find that the maxima of the entropy "landscape" change -- however it can be confusing because often the notation and language hides the fact that the entropy depends on extra pieces of information beyond those that explicitly appear in the formula above.

Moving away from a gas, one can calculate the entropy for many different kinds of physical systems, including the early Universe. However, one will need to think about the appropriate phase space, and in particular the appropriate measure used to count microstates (this is a particular challenge in early Universe physics). It should also be clear that entropy can be applied in non-physics contexts; for example computing the entropy of a text file amounts to evaluating probability distributions over certain combinations of letters. The notion of entropy will depend on how you define the space of "allowed" letters and words.

A further potential source of confusion is that in physics, there are certain simplifications or limits that are often made, that are completely correct in context. But, if a book uses these as the starting point (which is often a good idea pedagogically), the connection to the definition given above may not be clear. In physics applications, the probability distribution that maximizes the entropy assigns equal weight to all microstates. In this case, $p(x)=p$ is a constant independent of $x$. In fact we can write $p=1/\Omega$, where $\Omega$ is the total number of microstates. In this case, the entropy integral reduces to the usual "Boltzmann" form \begin{equation} S = - \frac{\log(1/\Omega)}{\Omega} \int {\rm d} x = \log \Omega \frac{\Omega}{\Omega} = \log \Omega \end{equation} since $\int {\rm d} x 1$ is just the total number of microstates.

Once you have this, you can use statistical mechanics to derive the typical rules that the state variable called entropy in classical thermodynamics obeys.

Answer 7

The cited definitions in your question actually focus on the evolution of entropy over time, not what is entropy itself. I mention this because it's important we keep these things distinct in discussion.

Probably anyone desiring to understand the concept of entropy - rather than simply memorizing the formulae - is confronted with the sentiment you express: The definitions commonly offered leave one unsettled, lacking intuition.

I cringe every time I hear someone use the popular disorder definition of entropy. As you say, this definition just leaves one unsatisfied with the necessary questions "What is order?" and "What is disorder?" It's turtles all the way down! Is a cluttered room really less ordered than a room we feel is straightened up? When people use this description, they are really attempting to describe what happens to entropy over time in a closed system - not what is entropy. i.e. There are many more ways for a room to be considered "messy" than straightened up. Therefore, a messy room has a significantly higher probability, and it will naturally become more messy over time without intervention.

I offer two references, which, in combination, will give abundantly satisfactory answers to all your entropy questions.

1. Enrico Fermi's Thermodynamics (very readily available)
2. Charles Kittel & Herbert Kroemer's Thermal Physics 2nd Edition, W.H. Freeman and Company, 1980

Fermi's Thermodynamics contains a wonderfully concise but complete derivation of the classical concept of entropy, along with deriving relationships between various forms of energy. It briefly mentions Boltzmann's derivation of $S = k\space log(\pi)$ where k is Boltzmann's constant, and $\pi$ is proportional to the probability of occupying a particular state. Without derivation of the latter, this still leaves one without intuition of what is entropy.

For the purpose of understanding what is entropy, the first (!) page, 2nd paragraph of Reference 2 suffices, with a perfectly succinct description of entropy: "The entropy measures the number of quantum states accessible to a system."

It goes on to say:

A closed system might be in any of these quantum states and (we assume) with equal probability. The fundamental statistical element, the fundamental logical assumption, is that quantum states are either accessible or inaccessible to the system, and the system is equally likely to be in any one accessible state as in any other accessible state. Given g accessible states, the entropy is defined as $\sigma = log\space g$. The entropy thus defined will be a function of the energy $U$, the number of particles $N$, and the volume $V$ of the system, because these parameters enter the determination of $g$; other parameters may enter as well. The use of the logarithm is a mathematical convenience: it is easier to write $10^{20}$ than $exp(10^{20})$, and it is more natural for two systems to speak of $\sigma_{1}+\sigma_{2}$ than $g_{1} g_{2}$.

When two systems, each of specified energy, are brought into thermal contact they may transfer energy; their total energy remains constant, but the constraints on their individual energies are lifted. A transfer of energy in one direction, or perhaps in the other, may increase the product $g_{1} g_{2}$ that measures the number of accessible states of the combined systems. The fundamental assumption biases the outcome in favor of that allocation of the total energy that maximizes the number of accessible states: more is better, and more likely. This statement is the kernel of the law of increase of entropy, which is the general expression of the second law of thermodynamics.

Now, how do we, in principle, get the number of quantum states? By counting, of course.

I mention a 3rd reference

3. F. Reif's Fundamentals of Statistical and Thermal Physics

for the purpose of capturing counting techniques in the first couple of chapters, and the H theorem derived in the Appendix which describes in terms of probability the evolution of entropy over time, along with rigorous derivation of fundamental concepts in statistical physics.

In essence, entropy is a measure of the relative number of possible states of a system. This is the common thread linking all examples you cite. That entropy must increase follows from the definition of probability: the more probable something is (the more states representing that something), the more often that something will occur in a dynamic system (as the system evolves through time); i.e. something will occur in direct proportion to something's probability - the evolution of entropy over time is simply a rewording of the definition of probability. To decrease entropy, an intervention is required, specifically tailored to achieve a state with lower probability. The intervening agent (no need to be sentient being) must gain more entropy (or produces, via heat) than it extracts, however.

Answer 8

Since the previous answers do a pretty good job explaining the concept, I will try to do the same albeit in layman terms -
Digression (can skip this part if you're familiar with the concept of 'accessible quantum states' in Statistical Physics) - On page no. $10$ of An Introduction to Thermodynamics and Statistical Mechanics by Keith Stowe, the total number of acessible states in a classical phase space is given by the very intuitive expression of -
$\Omega = \frac{V_r V_p}{\hbar ^3} $, where - $V_r$ = available volume in co-ordinate space, $V_p$=available volume in momentum space & $\hbar$=reduced Planck constant (btw, this is how quantum uncertainty naturally finds it's way into Statistical Physics).
Returning to the main question, on page no. 331, the author defines the total number of acessible quantum states as -
$\Omega = \exp{S_R /k} \rightarrow S_R = k (ln \Omega)\rightarrow(1)$
where - $S_R $ = entropy of the system & $k$ = Boltzmann constant.
Loosely speaking from equation (1), the entropy of a system is proportional to the number of accessible quantum states (this is what entropy is!) $\rightarrow$ more the amount of (co-ordinate/momentum) space available, higher the entropy, higher the disorder (because there will be more choices available for a particular particle to settle into - so, "order" does not mean a well-arranged/ordered outer appearance but refers to the number of accessible quantum states in $V_r$ & $V_p$), i.e., Just by looking at a system from outside, we cannot tell whether it has low/higher entropy than before because it's a very microscopic concept (no. of accessible states is a microscopic concept).
From the above discussion - even if you have a very constricted spatial volume available, it's entropy could still be deemed high if it has a lot of momentum/energy states vacant (as compared to an identical system but with most of its momentum/energy states already filled). Also, there's a proven law that entropy should always increase (a property it shares with 'time' which also moves in only one direction).
For your question on Big Bang (point nos. 2 & 3), the Big Bang doesn't violate the entropy law as the (co-ordinate) volume space ($V_r$)of the Universe has since (the Big Bang) only increased.
For snowflakes you say Because to me a snowflake is a perfect example of order. - "order" (as mentioned above) does not mean a well-arranged/ordered outer appearance but refers to the number of accessible quantum states in $V_r$ & $V_p$. In a snowflake, the ice entropy is actually higher because there are many different ways of ordering the protons (hydrogen atoms), if we know the number of Oxygen atoms in the lattice, hence, a snowflake has higher disorder (or entropy) as compared to water for the cold temperatures in which snowflake formation occurs., and for Point no. 4 on "Modes", what do you mean by uniform states that were the most abundant possible states.,how do you know they were most abundant, and what is meant by "uniform" here, please clarify?
Let's get to the definitions of entropy as mentioned in your question one-by-one -

• Entropy = disorder, and systems tend to the most possible disorder - explained above in the answer.
• Entropy = energy distribution, and systems tend to the most possible energy distribution - this essentially means a particular particle has higher probability to occupy that state which has higher frequency of occurence = eg. if a system has more spin-up ($|\uparrow \rangle $) states available than spin down states ($|\downarrow \rangle $), then the particle has higher probabilty to be spin-up rather than spin-down.
• Entropy = information needed to describe the system, and systems tend to be described in less lines = this sentence seems a little incomplete but would basically be interpretted to mean $\rightarrow $ more number of states $\rightarrow $ more information needed to describe a system (would need the preceeding + succeeding lines to properly understand this statement).
• Entropy = statistical mode, and the system tends to go to a microscopic state that is one of the most abundant possible states it can possess. = this is the same as described in the $2^{nd}$ point above.

P.S. - By a quantum state, we mean it's volume ($V_r = dxdydz$ & $V_p = dp_x dp_y dp_z$) cannot be known beyond the precision of $\hbar ^3$.