# Why does maximal entropy imply equilibrium?

From a purely thermodynamical point of view, why does that entropy have to be a maximum at equilibrium? Say there is equilibrium, i.e. no net heat flow, why can the entropy not be sitting at a non-maximal value?

From the second law of thermodynamics, it follows that $S$ never gets smaller and of course I know that for an isolated system there are many statements involving $\text{d}S=\frac{\delta Q}{T}$, which say what might happen for processes. But if I have equilibrium, then no relevant processes are going on. Some proofs in thermodynamics involve arguments how if we don't have maximal entropy, then we can do something which raises entropy. But why is that relevant or related equilibrium, i.e. to the positions for the termodynamic parameters, which don't change with time? One could argue that probably the energy $U$ doesn't sit at the minimal value, but in thermodynamics, without microscopic forces, the statement that the energy changes towards equilibrium and seeks its minimal value seems to be derived from maximal entropy.

And how can I conclude the converse? Why does equilibrium follow from $\text{d}S=0$?

(edit: I see the question was just bumped on the front page, and as it's a year old now, I guess my interest in the topic has changed in so far as I'm not particularly happy with the formulation of the initial question anymore. That is, I guess without proper stating the definitions, it might be difficult to give a good answer - the thing I still don't "like", and that might be kind of a language problem, is that "to maximize" implies that there is a family of values it could take - but it takes the maximum - while at the same time, you often consider a deviation away from a thermodynamical state to be a transition into a configuration where thermodynamics doesn't apply anymore. Hence, if you go away from the "maximal value", you might lose the concept of there being a temperature at all, but since this is what makes the parameter space with respect you use the word "maximal" entropy, you get into language problems. But at least I do see it's use in explainaing how the entropy/energy evolves once you take the extensive parameters into your control and change the system quasistatically. In any case, I am and was only only interested in a non-statistical mechanics answer here. Clearly, I can make sense of the physics using the microscopic picture anyway, but was purely interested in the formulation of the thermodynamical theory here.)

-
Sometimes "entropy is maximum in equilibrium" is just put into the formulation of the second law. So what is the exact formulation of the second law you are using? –  Yrogirg Jan 30 '12 at 17:41
The Clausius or Kelvin formulations at the beginning of the wikipedia page would suffice. The mathematical formulations involving $S$ are good as well. It's just that I don't see how a statement involving equilibrium, which is a concept involving time and time evolution, can be characterized in classical thermodynamics in terms or $dS$ and so on. –  NikolajK Jan 30 '12 at 18:02
@Nick Kidman Comparative statics has a concept of equilibrium but no dynamical theory. This reminds me of Joan Robinson's famous critique of neo-neo-classical economic theory's use of comparative statics. –  joseph f. johnson Jun 8 '13 at 16:20
@josephf.johnson: erm.. –  NikolajK Jun 8 '13 at 17:33
well, there's these things called "thermodynamic potentials"---I'm thinking about an answer to your question. –  joseph f. johnson Jun 8 '13 at 18:22

First, if ${\rm d}S\neq 0$, then the entropy will change, and because something is changing, it's obviously not an equilibrium.

If the physical system doesn't maximize the entropy and it's composed of many parts that may interact with each other, directly or indirectly (it's not disconnected), then any path that allows the entropy to be increased (given fixed values of conserved quantities such as energy) will be realized, so you will be away from the equilibrium because something will change.

If a system is composed of two or more decoupled, non-interacting components – like a bottle of blue lemonade and a bottle of red lemonade – they may be in equilibrium even if the entropy isn't maximized. One could increase the entropy by mixing the liquids but because they're not in contact, they won't be mixed.

On the contrary, if the entropy is already maximized, the only way how the system may evolve is to evolve into another state with the same, maximum value of entropy: there's no higher allowed value and the second law of thermodynamics prohibits a decreasing entropy. This is atypical because when we maximize entropy among all states with the same conserved quantities, the state of maximum entropy is typically unique. For example, if there are also movable macroscopic bodies that may create heat by friction, the entropy is maximized only when the friction stops the macroscopic motion and converts its energy to heat.

In all these discussions, one has to be careful on whether or not we're maximizing the entropy among all states or just the states with the same value of energy (and other conserved quantities). If we allow the energy to change arbitrarily, the entropy isn't really bounded from above (and discussions about its maximization are rendered meaningless) because any body may be heated to pretty much arbitrarily high temperature (or it may collapse into a black hole with an ever greater mass and therefore an ever greater entropy, too).

-
Okay, the second part basically says that if the maximum is unique, then the entropy has nowhere to go, so equilirbium. I still don't see why $\text{S}\ne 0$ implies that the entropy will change. Just becasue $S(U,V)$ is such that it grows with $U$ and $V$, why does this imply that $S$ will change with time on it's own? In classical mechanics, if $F=-\nabla U$ isn't $0$, then I understand that something will happen. However, I don't see how what in thermodynamics says that "if $\text{d}S\ne 0$, then the entropy will change". What says such a $(U,V),(T,P),...$ or whatever point is not stable? –  NikolajK Jan 29 '12 at 18:36
Dear Nick, as I said, I think that your statement that "something will change" is only true if the entropy is (locally) non-maximal among states with the same energy. Then it is guaranteed that the system will evolve in the direction in which the entropy increases - and free energy decreases. The force is really $-\nabla \Phi$, a gradient of the free energy, when one does it right, so this increases with the gradient of the entropy. –  Luboš Motl Jan 30 '12 at 7:06
But where in the laws of thermodynamics is this movement away from non extremal values encoded? I don't see that if thermodynamics is a theory which makes statements about equilibrium, how it handles points down at the math, which "are not yet" at equilibrium and tells them where to go. –  NikolajK Jan 30 '12 at 7:15
Dear Nick, the increase of entropy is the second law of thermodynamics. When the entropy isn't maximized, the second law may really be interpreted as a "strict inequality": the entropy strictly increases, so something has to be changing about the system because $S$, a function of the variables, goes up. That's why it's not an equilibrium. The actual rate by which $S$ increases depends on the situation but as I indicated, the force trying to move the system in the higher-entropy direction is really given by $F=-\nabla (E-TS)$, a gradient of free energy. It includes $+T\cdot\nabla S$. –  Luboš Motl Jan 30 '12 at 9:38
@Motl: I still don't see why the laws of thermodynamics can make statements about a change in time. I principally don't see how there can be a theorem about what will happen, if there is no time involved in the theory based on three axioms. The force relation $mx''(t)=F=-\nabla...$ isn't part of classical thermodynamics, is it? –  NikolajK Jan 31 '12 at 22:08

Let us focus on the case of a simple homogeneous substance that does not dissociate or undergo other chemical reactions. Then a thermodynamic state is any state where the substance has a definite pressure, equal in all its parts, a definite temperature, without any temperature variations in its different parts, and a fixed volume (it is not busy expanding or contracting). As usual we also assume the mass is fixed and so the density is also definitely related to the volume: so nothing more will be said about mass or density. A bit of, e.g., gas cannot be in equilibrium if different parts have different temperatures, this is an observational fact: the temperatures tend to equalise. But we just make it part of the definition of a 'state' that the system has a definite temperature. It is only in more advanced, not so classical, thermodynamics that one relaxes to some extent this assumption. This is not the definition of equilibrium, it is the definition of 'state'. Now further, the definition of equilibrium is that if there is any thermal contact with the environment, then the environment must also be at a definite temperature, the same as that of the system. If the system can interact with the environment through doing work by expanding its volume, then the pressure of the system must be equal to the pressure of the environment. This is the definition of equilibrium. Note carefully: if there is no interaction with the environment, then every thermodynamic state is a state of equilibrium, because there is no condition that it has to satisfy. All of these remarks are logically prior to either the First or the Second Law of Thermodynamics, prior, too, to notions of reversibility or irreversibility, even to the notion of 'path'. For that reason, they are sometimes called 'The Zero'th Law of Thermodynamics'. All of this makes sense even if one does not know that temperature or pressure is the result of the motion of molecules: it was formulated before the modern theory of heat was understood, it was sometimes thought that heat was a fluid.

Comparative Statics does not have a true dynamics, but it is meant as a shortcut for use when the true dynamical evolution of the system is either unknown or too complicated to analyse. One defines `equilibrium' by the balance of forces, by having the net sum of the forces acting at any point be zero. In thermodynamics, the two main principles are equalisation of temperature, thought of as the result of transferring heat from the hotter body to the colder, and equalisation of pressure, which occurs by expansion or contraction of the volume. It is assumed that if one studies the properties of a system at one equilibrium point (i.e., the equilibrium state which would subsist under one set of values of the parameters such as the temperature of the environment and the pressure of the environment), and then studies the properties of that same system at a different equilibrium point which would subsist under different values of the parameters, then one has a good deal of information even about the process of transformation from the first point to the second point. In thermodynamics this is true provided the transformation proceeds slowly enough.

Equilibrium is meant to embody the notion that the system is in a stable state and will not evolve. Even so, the notion in thermodynamics of a reversible path is that a system passes through several different equilibrium states under outside influence (namely, a change in the external parameters which means that what was formerly an equilibrium state is no longer in equilibrium with the environment). Although it may violate strict logic, this is a very useful idealisation of reality. (In reality, only irreversible processes can take place.)

All of Classical Thermodynamics is Equilibrium Thermodynamics (to a first approximation), but it studies heat engines and refrigerators, in which the system does pass through different states, changing its pressure and volume. The theory uses comparative statics to be able to make statements about the end results of these processes, without being able to say anything about how the process happens exactly, at what rate, or anything truly dynamical.

If the system is in the state given by an equilibrium point, it will not change its state unless the external conditions are changed. If the external conditions, such as the temperature of the environment, go through a finite non-zero change, the shock will lead to an irreversible transformation through disequilibrium states, e.g., the part of the system closer to the boundary will heat up before the rest of the system, and this means the system is not in a thermodynamic state at all. Eventually the system will reach a new equilibrium state, but the amount of work done or heat absorbed during this transition period is not exactly calculable using Classical Equilibrium Thermodynamics. (At least, not in the elementary part.)

If the non-zero change is very small, it will be a reasonable approximation to treat it as the result of an infinite number of infinitesimal changes. An infinitesimal change does not shock the system out of equilibrium, but coaxes it gently to a neighbouring state of equilibrium. Hence the system traverses a reversible path of equilibrium states. Infinitely slowly. In reality this is not possible. Logically, if a path is reversible, then there is no reason for the system to pick which direction in which to traverse it, so the system will stay put. But this is an idealisation which is a useful approximation to the real world of irreversible transformations through disequilibrium states. More importantly, it gives us a theoretical bound on the efficiency of an irreversible process, an efficiency which can never be improved on.

The Second Law of Thermodynamics will be taken in the formulation due to Lord Kelvin and preferred by Nobel laureate Max Planck. It is impossible to find a cycle (of paths) whose only effects on the environment are to produce work and take heat from the environment.

Note by contrast that the Carnot cycle both takes in heat and puts out heat. In fact, it has to have two heat reservoirs, at two different temperatures. It takes in heat from the hot one, and puts out heat to the cold one. The point of the Second Law is that it is impossible to turn into work all the heat taken from the hot reservoir. Some of the heat has to be 'passed on' to a colder reservoir.

Our strategy will be to first consider reversible cycles. A reversible path is one where we infinitesimally nudge one equilibrium state to a nearby one and there are only two ways to do this: we can put it into thermal contact with an inifitesimally hotter (or colder) heat reservoir. It will absorb (or lose) a tiny amount of heat so slowly that the equilibrium is not disturbed, and any other adjustments allowed by the setup will also happen infinitely slowly and delicately. We can adjust its volume infinitesimally: we can alter, very delicately and infinitesimally, the pressure of the environment which, if we allow the system to be in the kind of setup where it feels the pressure and can alter its volume, it then contracts (or expands) until pressure is equalised.

There are only two ways to reversibly move the equilibrium state around, any reversible path is really a combination of adiabatic transformations and isothermal transformations. If we allow thermal contact with the environment, that is of course an isothermal transformation. If we do not allow thermal interaction with the environment, then of course no heat can be exchanged, and that by definition is an adiabatic transformation, and that is the only way the temperature can be altered.

The first point is to deduce from Lord Kelvin's formulation of the Second Law of Thermodynamics that no cycle (heat engine) can be more efficient than a reversible cycle (heat engine).

The basic idea of this deduction is that if C were a cycle (and, for simplicity, assume it operates between two heat reservoirs only, 1 is 'hot' and 2 is 'cold') that was more efficient than a reversible cycle R operating between the same two reservoirs, then the Second Law would be violated. For we could run the engine C and use its external work to drive R in reverse, as a refrigerator, since R is reversible. By scaling R up or down, if necessary, we can assume that R requires the same amount of heat input from the cold reservoir as is output there by C. Since C is more efficient than R, C produces more work than R really needs to operate. Hence, the combination of running C then R is that the cold heat reservoir is unchanged, heat has been absorbed from the hot reservoir only, and net work has been produced (since not all the work produced by C was needed by R). But this violates the Second Law of Thermodynamics.

The second point is to note that therefore, every reversible cycle operating between the same two temperatures has the same efficiency. For we can apply the above argument to two reversible cycles, and then neither of them can be more efficient than the other.

We are about to use a little algebra, so we have to formalise the notation for the intuitive argument we already presented in words. Let $W$ be the total work done in one cycle by a given engine. Since it is a cycle, there is no change in the internal energy $U$, so the work done is $W = \Delta Q_1 - \Delta Q_2$, where $\Delta Q_1$ is the heat absorbed at the hot reservoir and $\Delta Q_2$ is the wasted heat, expelled to the cold reservoir. The efficiency, by definition, is $W \over \Delta Q_1$ and this is equal to $1-{\Delta Q_2 \over \Delta Q_1}$. Hence, all reversible cycles operating between the same two reservoirs have the same ratio $\Delta Q_2 \over \Delta Q_1$.

This fact is used to define the temperature of the reservoirs by $${\Delta Q^R_2 \over \Delta Q^R_1 } = {T_2 \over T_1} .$$ (I wrote $R$ this time to emphasise that this is only for reversible cycles.) This defines the temperature up to a scaling factor. It also immediately gives us $${\Delta Q^R_2 \over T_2 } = {\Delta Q^R_1 \over T_1} .$$

Now, redefine $\Delta Q_2$ to mean, just as with the other reservoir, the heat absorbed by the system from the cold reservoir. This is more rational, less prejudiced against refrigerators. We now get for all reversible cycles, $${\Delta Q^R_2 \over T_2 } + {\Delta Q^R_1 \over T_1} = 0$$ and for an arbitrary cycle, $${\Delta Q_2 \over T_2 } + {\Delta Q_1 \over T_1} \leq 0 .$$

By approximating an arbitrary path by segments like these and using lots of reservoirs, two for each segment, we get the curved versions:

$$\int_{\mbox{closed path}} {dQ \over T} \leq 0,$$ with equality whenever the path is completely reversible. This is Clausius's inequality, and it follows from the Second Law by the line of reasoning we followed involving comparing an arbitrary cycle to a reversible cycle being run in reverse. Arranging it to cancel out the effect of the given cycle on one heat reservoir, and getting a contradiction to the Second Law.

If we now confine ourselves to reversible transformations between equilibrium thermodynamic states only, then the fact that this path integral is zero means we can define a potential function $S$ (well-defined up to an additive constant) such that $$S(\mbox{state two}) - S(\mbox{state one}) = \int_1^2 {dQ \over T}.$$ This is pure mathematics. It's like an antiderivative (except in two dimensions). This is the basic definition of entropy in Classical Thermodynamics (as distinguished from the informational definition of entropy in Stats or in Stat Mech, due to Boltzmann, Sir Ronald Fisher, and Shannon).

But from the Clausius inequality, it follows that for an irreversible path, $$\int_1^2 {dQ \over t} < S_2 - S_1 .$$ Now, for an isolated system, $dQ$ is always zero since no heat can be exchanged with the environment. So $$S_2 - S_1 > 0.$$ I.e., along an irreversible path, the entropy of an isolated system increases, and along a reversible path, the entropy cannot change. There is something illogical about this. Entropy has only been defined for equilibrium states, and if a system is isolated, there is no way to disturb the system, and so, no processes will take place.

But as an idealisation, to give us simplified approximations to real processes provided they take place very slowly, it is logical in its own way. Clausius's inequality does not tell us how fast the dynamics will go along a path, but it will tell us which direction is possible and which direction is impossible. Because $$\int_1^2 {dQ \over t} = - \int_2^1 {dQ \over t} ,$$ so if one direction yields a negative value of the integral, the other direction will yield a positive value, which is forbidden by Clausius's inequality so that other direction is impossible.

-

This is now Part II.

PART II entropy and disequilibrium

I will explain the ideas from a footnote in Fermi, Thermodynamics, p. 50, on entropy in disequilibrium.

There is the idea, an imperfect one, that 'entropy is additive'. If two systems each undergo a change in entropy $\Delta S_i$, then the combined system undergoes a change in entropy $\Delta S_1 + \Delta S_2$. Fermi comments that this is not always valid, but is if 'the energy of the system is the sum of the energies of all the parts and if the work performed by the system during a transformation is equal to the sum of the amounts of work performed by all the parts. Notice that these conditions are not quite obvious and that in some cases they may not be fulfilled.'

I already remarked in Part I that if a substance does not possess one uniform temperature throughout its volume, it is not even in a thermodynamic state. But there is something we can do about this provided there is no turbulence: provided the variation in temperature in space is smooth, we can divide the substance into many small subsystems and make the approximation that the subsytems are each at a uniform temperature and pressure. Can we define the entropy of each small subsystem?

Assume further that we can treat the entropy as additive. To do this, we have to assume each small piece is in equilibrium, so it cannot be in thermal contact with the other subsystems. So imagine that each small volume is divided from the rest of the region by little non-conducting walls, which are also immovable.

For simplicity, let us consider only two smaller regions. Imagine two boxes of one cubic light year each are filled with hydrogen gas and placed side by side. The left box is at equilibrium at temperature $T_1$, the right one at the higher temperature $T_2$, and each box contains the same mass of gas. Each box is isolated from the environment and from the other box. Then each one is in equilibrium and the combined system is also in equilibrium. We can measure changes in entropy from this state as a starting point, state one in the integral formulas from Part I.

If we replace the dividing wall by a perfect thermal conductor, the states are no longer in equilibrium and an irreversible transformation will take place until the temperatures are equalised. We will call this state two. But in order to use our definition of entropy, we have to think of how to connect state one with state two by a reversible transformation that takes place through equilibrium states. We can do this. It is impossible, but still physically meaningful.

Replace the dividing wall by a perfect thermal conductor wall only for an infinitesimal period of time, then put the insulating wall back. This is a reversible transformation to a state where the left box is in equilibrium at temperature $T_1 + dT_1$, and the right box has temperature $T_2 + dT_2$. The pressures will have changed a well since the volume has been constant. If these are perfect gases, one can show that $dT_1 = - dT_2$ by conservation of energy, but never mind. This is not really a dynamical analysis since time as a variable is not being considered. Note that this reversible transformation could also be brought about with two external heat reservoirs instead, leaving the dividing insulating wall in place. In Part I we proved that the exact details of how a reversible transformation between two equilibrium states is brought about make no difference to the value of the entropy change.

Repeat this process of replacing the insulating wall by a conducting wall briefly, allowing the equilibrium states of the two boxes to be nudged to neighbouring equilibrium states with closer temperatures. Continue until no further change occurs. Obviously by symmetry, this will be for the average temperature $T_1+T_2\over 2$. Or conservation of energy since no work is done, the change in heat is the same as the change in internal energy $U$, and $dQ=c_vdT$; for a perfect gas $c_v$ is a constant.

This is a reversible transformation. We can return to state one. We use two heat reservoirs. Leave the dividing insulating wall in place, put the left in contact with an infinitesimally cooler reservoir and the right in contact with an infinitesimally hotter reservoir, repeat as needed until state one is reached. Because the dividing wall was non-conducting, each state traversed is a state of equilibrium.

Therfore we can calculate the change in entropy of each box, add them up. The formulas from Part I are $$S_2-S_1 = \int_{T_1}^{T_1 + T_2 \over 2} {dQ_1\over T_1} + \int_{T_2}^{T_1 + T_2 \over 2} {dQ_2\over T_2}.$$ In general, $dQ = dU + dW$ where $U$ is the internal energy and $W$ is the work done. Here, no work is done, and by energy conservation, $dU_1 = - dU_2$. In fact, by the perfect gas law, $dQ = c_vdT$ so we get $$S_2-S_1 = \int_{T_1}^{T_1 + T_2 \over 2} {c_v dT_1\over T_1} + \int_{T_2}^{T_1 + T_2 \over 2} {c_v dT_2\over T_2}.$$ Performing the integrations, this becomes $$S_2-S_1 = c_v\log{T_1 + T_2 \over 2T_1} - c_v\log{T_1 + T_2 \over 2T_2}.$$ Now this transformation took place in thermal isolation from the environment. Planck proves the theorem that 'If a system of perfect gases pass in any way from one state to another, and no changes remain in surrounding bodies, the entropy of the system is certainly not smaller, but either greater than, or, in the limit, equal to that of the initial state; in other words, the total change of entropy $\geq$ 0. The sign of inequality corresponds to an irreversible process, the sign of equality to a reversible process.'

One can indeed check that the sign of this change in entropy is positive if $${(T_1+T_2)^2\over 4T_1T_2} > 1,$$ i.e., if $$(T_1+T_2)^2 > 4T_1T_2,\,\, \mbox{, i.e., if }\,\, (T_1-T_2)^2 > 0$$ which is elementary. Its minimum is when $T_1 = T_2$, that is the only case where the change in entropy is zero, hence that is the only case when the entropy is maximal among all these equilibrium states with constant energy, connected to state one by a reversible transformation.

In order to use the definition of entropy we had to show how to reverse the transformation from state one to state two but we did not have to reverse it while remaining in isolation. If we remain in isolation, all we can manipulate is the dividing wall, we cannot use heat reservoirs. While the system is in isolation, then, it moves through equilibrium states in a manner which is reversible in one sense, but irreversible in another sense. Therefore the entropy has a maximum when the temperatures are equalised, which is the only stable equilibrium. It is the only state which will not evolve further no matter what we do with the dividing wall. The other equilibrium states become non-equilibrium when the dividing wall is allowed to conduct heat. This is the sense of the slogan, 'equilibrium is when entropy is maximised.'

It is also true that even this stable equilibrium state would become non-equilibrium if put into contact with an external heat reservoir, which is how we showed we could reverse the transformation and get back to state one. But then the system is no longer isolated, but there is no law that entropy increases for a non-isolated system.

-
Thank you for your answer, I'll study it in detail when I get to it! –  NikolajK Jun 23 '13 at 21:35

"entropy is maximized" does not make sense as a dynamical statement in thermodynamics (in the usual definition). Entropy is constant in td at the equilibrium for fixed boundary conditions (dS=/=0 makes sense in td, for slow processes under a change of boundary conditions, but that is not directly related to the "entropy is maximized" statement).

Here is a way to make sense of the "entropy is maximized" statement:

1) Consider a set of fixed td states without transitions. E.g. in the previous lemonade example, consider a sample of fixed td states, with different mixtures in the two bottles, such that there is equilibrium in each bottle. The total entropy is maximal on perfectly mixed configurations (you could measure the difference by a process like in 2)). So entropy is maximal for the perfect mixings on the set so defined. But the system stays in any td state with lower entropy as well.

2) Now allow a dynamical transition between the configurations in 1). The non-perfectly mixed states will develop to the perfectly mixed states, which are the equilibrium states. Entropy is maximized dynamically on the set of n-e states. The entropy function which is maximized here is strictly speaking no longer the td entropy, as the latter is not defined except for equilibrium states. It is well-defined in statistical physics, as logarithm of the multiplicity of the n-e state.

1) is not a dynamical statement, 2) takes one outside of td. There is no dynamical maximization of the td entropy, but of a minor generalization of an entropy function defined on n-e states.

-
thanks for the response. –  NikolajK Nov 12 '12 at 9:05

Thermodynamics is a science that happens to be concerned with extremely complicated systems that can be analyzed with only a few variables. For example, the gas in a pressurized bottle consists of something like $10^{23}$ atoms, each with a position (3 variables) and a velocity / momentum (3 more variables). That's a lot of variables, but thermodynamics gives us a way of analyzing the system (I'll use the energy formulation) with just volume, total energy, and number of atoms. From this we get the extensive values pressure, temperature and chemical potential (this last is not needed for the fixed amount of gas in a bottle), and other things such as heat capacity, etc.

In making an analysis of these systems, we find that there is another variable that is very useful to know, the "entropy". From a microscopic point of view, the entropy is the logarithm of the number of states that are possible for the system. By "number of states" we mean the number of possible positions and momenta for those $10^{23}$ atoms.

The reason entropy is a useful thing is that it tells us how "easy" it is to set up the atoms in a particular situation. If there is only one way to do it (for example, all the atoms sitting next to each other in a crystalline solid at the bottom of the bottle), then that will be very difficult to achieve. On the other hand, if there are "billions and billions" (never mind exactly how many) ways of assembling that situation (as far as total energy, volume, and number of atoms goes), then that situation will be easy to achieve.

Sometimes you can get a system where the entropy is very small, compared to its maximal value. An example is a bottle with all the (ideal) gas on one side of the container. Such a situation is not in equilibrium because there are so many ways it could reorganize itself that it is doomed to change to a situation of higher entropy. Hence S must me maximized.

Now let me give an example of a situation where the numbers are extremely small but you'll still get an idea how the math works.

Suppose we had a handful of 6-sided dice, say $N$ dice. A given situation is that each die shows a number in {1,2,3,4,5,6}. A thermodynamic variable for the handful is "the total of all the numbers showing on the dice." We suppose that the dice have an interaction which causes them to randomly change their orientation (and hence their numbers). For $N$ dice, the average (or expectation value) for this thermodynamic variable is $N$ times the average number on a die which happens to be 7/2. Thus, on average, the dies will add up to $7N/2$.

Suppose the dies happened to be in a situation where their numbers summed up to $N$ (or $6N$). Such a situation has only one way of being achieved -- all the dies have to show the same number, i.e. "1" (or "6"). Such a situation has an entropy of log(1) as there is only one way it can be achieved, and the entropy is $\ln(1) = 0$. This has an unnaturally low entropy and is not an equilibrium situation.

On the other hand, there are many ways of getting the dies to sum up to $7N/2$, (at least supposing you have an even number of dice!) Thus this situation is one with high entropy. So a handful of jiggling dice, tends to approach thermodynamic equilibrium by exhibiting a total value that is equal to the sum of their average possible values.

-
Thanks for the response. The point is that I'm rather looing for an explaination "from a purely thermodynamical point of view". A systematic derivation from the laws of thermodynamics would suffice, however I guess a clear definition of equilirbium will have to be stated as well. –  NikolajK Jan 29 '12 at 19:03

Although the previous answers are well articulated i would like to give an alternative and more intutitive viewpoint very shortly from two inter-connected angles:

1. By the 2nd Law, the entropy of a system increases or becomes a maximum i.e $\Delta{S} \ge 0$. This means that once $\Delta{S}$ reaches a stationary point (maximum entropy) the system is in equilibrium.

2. By Boltzmann's (aka statistical mechanics) relation, entropy is analogous to the number of configurations ($S = k_BlogW$) in that state of energy. This implies: maximum entropy $\implies$ maximum number of configurations. This means that if the system is (slightly) perturbed from this state, it has a larger amount of available configurations to cope for this perturbation (or is more likely to be stable). This can be related to similar results of KAM theory for hamiltonian systems.

Regarding the relation between maximum entropy state and equilibrium state of a dynamical system. Consider the equilibrium state as a state of a system where no more work can be extracted (this is again a generality in the spirit of the question), as such it is also a maximum entropy state.

Conversely in maximum entropy state, no more work can be extracted, thus it is also an equilibrium state.

Consider a dynamical system which can achieve a number of configurations at a specific energy. In the maximum entropy state, all the available system configurations are already used by the system and this represents the cancellation of any potential differences with the system environment. Thus (at that energy state of the system) no more work can be extracted and is in (dynamical in general) equilibrium (with environment).

Conversely, a dynamical system at equilibrium (with environment) (at a specific energy), has null potential differences (with environment) thus no more work can be extracted and this is in maximum entropy.

The analogy is very similar to the least action principle of mechanics (which actually is not necesarily a least action but a stationary action). Why should least action give a dynamical law?

(the answer is somewhat similar to both questions, although a deeper investigation can make things more clear and inter-connected)

-
Thanks for the response. Though my comment to Carl Brannens answer applies here too: In the first sentence of the question I say I consider the question from the point of view of pure thermodynamics, not arguing using the statistical physics model of it. –  NikolajK Jun 21 at 21:29
@NikolajK, using a thermodynamics picture is the same (as the 1st argument), i.e system will evolve dynamically towards a state of maximum entropy. Using stat. mechanics gives another formulation. Also temperature can also be used. Take a look at Generalized Thermodynamics book by Keenan, Hatzopoulos for generalisations of the 2nd law even for non-equilibrium physics –  Nikos M. Jun 22 at 10:42
@NikolajK, i fully understand the question, since i have similar questions regarding 1st principles understanding. The thing is that thermodynamics (although having thermo- prefix), is extremely general, do not asssume anything (a-priori) about tha constitution of the system, or the nature of particles (if any) or constraints etc.. In this sense 2nd law is a (general) dynamical law (it states the way or state sth will evolve). The thermo- prefix actually signifies that this investigation was triggered by heat engines, however it is stated and related to dynamics in general terms –  Nikos M. Jun 22 at 10:53
@NikolajK, edited answer to reflect the comment –  Nikos M. Jun 23 at 2:10