I have come across Liouville's theorem in the first chapter of many statistical mechanics textbooks, still I don't quite get how it is important to statistical mechanics.
How is it related to statistical mechanics? How can it be applied to the study of ensemble theory or other area in statistical mechanics?
Because you do equilibrium statistical mechanics. In the usual ensemble theory we associate to a system (a macrostate) a big number of corresponding microstates, each microstate is a point in phase space, that point is called representative point. Now you want to study how these points move in phase space.
First of all the situation of equilibrium for the system is the situation where the system is represented by a stationary ensemble. Stationary means that the density function $\rho(q,p;t)$ of the representative points does not depend explicitly on time:
$$\frac{\partial{\rho}}{\partial t}=0 \tag{1}$$
Now, Liouville's theorem tells you that the local density of the representative points, as viewd by an observer moving with a representative point, stays constant in time:
$$ \frac{d \rho}{d t} = \frac{\partial{\rho}}{\partial t} + [\rho,H] =0 \tag{2} $$
Where the last term is the Poisson bracket between the density function and the hamiltonian.
To satisfy both $(1)$ and $(2)$ you need $$[\rho,H] =0 $$
So Lioville's theorem somehow tells you that if you want to do equilibrium statistical mechanics, the Poisson bracket between the density function and the hamiltonian has to be null. It is a requirement that has to be satisfied, if it isn't, you're not doing equilibrium statistical mechanics, you're doing something else.
It is extremely important if you want to understand the reason behind the irreversibility paradox. Basically, classical mechanics are microscopically reversible. Then, why don't we ever see macroscopic reversibility for certain experiments? Why can we see a glass shatters but pieces do never join back into the original glass? Liouville's theorem explains it.
It states that the (hyper)volume in the phase space is conserved.
Let $|\Gamma|$ be the size of the set, namely $\iiiint_V \prod_{i=1}^{N} dq_i dp_i$ Where $q_i$ are the generalized coordinates and $p_i$ their momenta.
Think for example in the case of a vessel with a wall in the middle. One side is full of gas while the other half is empty (situation 1). Now let's take the wall away so that the gas can expand to the other side (situation 2).
Now there's twice the available space, thus the corresponding coordinate can take twice its values. The other coordinates remain the same and momenta are restrictionless again.
This is okay for one particle. Now, for the unthinkable number of particles in 1 mol of gas ($\sim 6\cdot 10^{23}$ particles), twice that number is just huge.
So the "size" of situation 1 is so small against situation 2 that it is totally negligible.
Conclusion: macroscopic reversibility in those cases IS possible, it is just not probable at all, since $\Gamma_1|/|\Gamma_2|\ll$.
