How is $\frac{dQ}{T}$ measure of randomness of system? What is entropy really? The answers to these questions sum up the thermodynamic meaning of entropy. But, I have seen way too many documantaries and YouTube videos explaining that entropy is 'dis-order'. Others say it isn't. Then there is the thing about entropy and information. So, what the heck is 'entropy', in the modern sense?
Note that I would prefer a layman's explanation and not one with the level of technicality of the above answers.
So, you want to know about entropy. Well, in thermodynamics it is defined as the measure of how much energy or heat is 'spread'. Then there is the second law:
Entropy always increases a.k.a. energy tends to spread out over time. - Second Law of thermodynamics
But, in a modern sense, we don't think of entropy just as a thermodynamic quantity measuring the spreading of energy. We define it to be a property of the arrangement of particles. This is where the idea of 'entropy is disorder' comes from. We will see why that is not quite true. Entropy is also connected to information in a way that eludes many. So, here we will explore the meaning of entropy, as seen by physicists today, but first, we need to cover up some terminology.
Microstates and Macrostates are usually described with checkerboards and chess pieces, but we will take a different approach to it. Checkerboards are discrete; while in any usual circumstance, we use continuous media.
So, let's imagine a football field: with some footballs on it. Now, let's say that all the footballs are stuck in some corner of the field. Let's also say that they are fixed in position and cannot move.
If you take a bird-eye view of the entire field, then you would see that all the fish are arranged in an area in the lower-left. Now, you call this macroscopic arrangement of the footballs a macrostate. That means that when you look at the system (field), the components (footballs) are arranged in one particular state. Remember, that you are concerned about the overall state of the system, not the state of the individual particles.
Now, let's zoom in. You may notice that there is a very particular arrangement of footballs in the area. Every individual football has a particular position; in jargon, every football has a precise, particular state. Now, if you swap the positions of two footballs, you have changed the individual arrangements of the footballs.
We call this particular arrangement of the individual components (footballs) of a system (field) the microstate. Now you are concerned about the individual states of the components of the system.
But, notice that when we changed the microstate by swapping positions, we did not change the macrostate. The footballs are still inside the same area in the same corner. Any macrostate can be found in a large number of microstates. This is an important point so we emphasize it:
Every macrostate can have a large number of microstates. When you change the microstate, you don't necessarily change the macrostate. The overall arrangement of the system can be formed by various different individual arrangements of the components.
Okay. Now, we move on to define entropy.
Consider the following situation:
We can see that the gas is compressed in the left half of the container. Now, that is a microstate of the system as a whole. The individual gas particles possess some state and energy, so we also have one microstate.
Now here is the catch: the macrostate in the picture has a lesser number of microstates associated with it than the macrostate where the gas fills the whole container.
This is not because of convention or the way we define microstates, this is a naturally occurring observation. We have more number of ways to individually arrange particles and their energies in the second macrostate than in the first.
Now, we can finally re-define the Second Law:
Systems tend to occupy those macrostates which have a higher number of associated microstates. - Second Law of thermodynamics, redefined.
That's it. Systems tend to shift to those macrostates which have a higher number of possible microstates. In this context, entropy is defined as: $$S = k \log(\Omega)$$ where $k$ is known as Boltzmann's constant and $\Omega$ is the number of microstates for a particular macrostate. We have turned something defined as a 'spread' of energy into something deeply related to the arrangement of the system itself.
The second macrostate (which I will call 'full') has more entropy than the first ('half'). So, the system tends towards a state of more entropy, which correspond to tending towards a macrostate with a higher number of microstates.
So, why does your coffee always cool down and never heat up if you leave it alone? Because the equilibrium state (temperature of coffee and surroundings is same) macrostate offers more possible ways to arrange energy than the hot coffee macrostate (coffee states hot and surroundings are cooler compared to coffee).
Note that in this description of arrangement of energy, we use statistics. Actually, this whole field is called 'statistical mechanics'. So, when you view at the situation statistically, it is possible that the entropy of a system can come down. But the probability of this outcome is so bleak that we often neglect it like it does not even exist. It would take much longer than the age of the universe for your coffee to get hotter if you keep it alone. But still, it is a nice point to keep tucked away in your minds.
I wont talk much about this one. Entropy and its connection to information is pretty much summed up in this answer: Entropy and Information
First we need to set some basics right. Lets talk about information. If I have a coin and I flip it, and I cover it up, you cannot know whether the coins shows heads or tails. It could be in any one of the states.
Now, entropy in information theory is defined to be the lack of information that you have about a system. You can alternatively define it as the information needed to describe the arrangement of each and every component of the system. But these two definitions are equivalent.
Now, if I open up the over or tell you the state of the coin, the you will get some information about the coin. How much information do you get quantitively? We say 1 bit, because you are getting to know only some information about one state of the coin.
That is all I will say with regards to information theory. You can read the brilliant answer linked above. We move on the 'disorder' stuff.
What about the most common thing you hear about entropy? Is entropy actually a measure of the disorder in a system? Many would tell you that that view of entropy is rather misleading. Here I would like to discuss why.
The stem of the pop-science idea 'entropy is disorder' comes from the fact that entropy is related to the arrangement of particles. Now, when you have a very disordered system, an uneven, random system, there are a large number of microstates for that corresponding macrostate. Think about it. Going, back to our example, if you can arrange some footballs anywhere on the field (which is a more disordered macrostate), then you have a large number of possible coordinate and states where you can arrange them. So, the entropy of a disordered state is more.
On the contrary, if you keep the footballs neatly arranged in a corner, there are lesser states in which you can arrange them. So, voila! Ordered macrostate, lower entropy!
That is where the entropy-as-disorder comes from. But, just because the entropy is greater in the disorder state than the order state, that does not mean that entropy is disorder. It is just a consequence of the definition of entropy which we looked at before.
So, is it correct to think about entropy as disorder? not exactly. Disorder means more entropy, but entropy is not disorder.
What about entropy as randomness? Again its the same thing! It stems from the fact that more entropy means more ways to arrange individual components, which may lead to the state being more random. But, by no means is entropy 'defined' to be randomness. It is just a pop-sci generalization of a deeper and fundamental concept.
This was an answer aimed at helping people understand what entropy actually stands for. Now, I think you may view the 'entropy as disorder' and 'entropy as randomness' as just simplification of a more fundamental concept. While I have glossed over some (read many) technical details, this is the meaning of entropy, defined in simple terms with analogies.
Now, you see this quote with more insightful thoughts:
If your theory is found to be against the second law of thermodynamics, I give you no hope; there is nothing for it but to collapse in deepest humiliation. - Arthur Eddington
Well said indeed!
Entropy is basically defined as a measure of a molecular disorder or randomness of a system. It is the measure of the system's thermal or heat energy per unit of temperature which is unavailable for doing work.
