Argument that entropy must always stay constant Entropy in information theory is $H=-\sum_ip_i\log(p_i)$. Suppose I have some distribution $p_t$ (e.g. uniform) over a finite set of $N$ microstates that the system is in at time $t$. Call the state of the system $s_t$.
Now assume that the laws of physics are reversible, then the relation between microstates at time $t$ and at time $\tau$ is a bijection. Hence, I can derive a new distribution $p_{t+\tau}$ over microstates, by simply calculating for each possible microstate $s_t$ what the state $s_{t+\tau}$ would be if the system evolved, and the entropy of $s_{t+\tau}$ would equal that of $s_t$, since we are effectively just swapping index numbers in the formula for $H$.
Hence, in a closed system with reversible laws of physics, entropy must always stay constant. This conclusion must be wrong, because it contradicts the second law of thermodynamics. What is wrong with this argument?
 A: This is a profound question in physics and does not yet have a universally agreed answer. The standard way to think about it is as follows.
Adopt the example of particles in a gas, and use phase space (position-momentum space) to track the system state. At some initial time the particle states fill some blob in phase space (classical physics suffices here; quantum leads to similar conclusions). As the system evolves, this blob maintains its area if the evolution is reversible. However the blob becomes extremely contorted. It soon gets distorted into very complicated and intricate twists and turns of structure, so that it is not a practical possibility to delineate exactly which parts of phase space are inside this complicated region occupied by the particles, and which are not. But the macroscopic properties don't care. So we might choose just to put a simple envelope around the outside of this complicated shape, and say "oh well, now the particle states are inside this envelope, and that is all the rest of the universe cares about when it interacts with this system." This simple envelope is, obviously, of larger area that the original blob, and so the decision to use it as the new statement about the state of the system is a decision to say that the system now has larger entropy than it had before.
My use of the word "decision" there looks as though the increase of entropy is the result of a human decision. If one regards entropy as a statement about ones knowledge of a system then that would be fair enough. However I prefer to think of entropy as a property of the system, and then what is being asserted in the increase in entropy is the statement that the rest of the universe does not, and will never, evolve in a way which depends on the details of the phase space structure of the set of states occupied by the particles. When one says "never" one is here making a physical statement which probably cannot be proved to be true; it is more a case of being a very reasonable bet. But aren't all "laws of physics" just very reasonable bets?
