Why does critical damping return to equilibrium faster than overdamping? How can one prove that the critical damping regime returns to equilibrium fastest? Is it true that there are cases where the heavy damped solution will return faster?
Consider a free harmonic oscillator $$x''(t) + \gamma x'(t)+ \omega_0^2 x(t) = 0.$$ This has solutions of the following:
Heavy damping: $\gamma > 2 \omega_0$
$$
x(t) = A_1 e^{(-\gamma/2-\sqrt{\gamma^2/4-\omega_0^2})t} + A_2 e^{(-\gamma/2+\sqrt{\gamma^2/4-\omega_0^2})t} = A_1 e^{-\mu_1t} + A_2 e^{-\mu_2t}, \tag{1}
$$
where
$$
\mu_1 = \frac{\gamma}{2}+\sqrt{\frac{\gamma^2}{4}-\omega_0^2} \text{  ,  } \mu_2 = \frac{\gamma}{2}-\sqrt{\frac{\gamma^2}{4}-\omega_0^2}.
$$
Clearly, $\mu_1 > \gamma/2 > \omega_0$ (since heavy damping configuration) and $\mu_1 \mu_2 = \omega_0^2$ so therefore $\mu_2 < \omega_0$.
Critical damping: $\gamma = 2 \omega_0$
$$
x(t) = (B_1+B_2t)e^{-\gamma/2t} = (B_1+B_2t)e^{-\omega_ot}. \tag{2}
$$
In $(1)$, the second term is the slowest to decay since $\mu_1 > \mu_2$. Furthermore, the second term in $(1)$ decays more slowly than the exponential term in $(2)$ since $\mu_2 < \omega_0$. Hence, critical damping returns to equilibrium faster than heavy damping.
However, what if the constant $A_2$ is zero? Surely then the heavy damping scenario returns to equilibrium more quickly than critical damping?
 A: It is only reasonable to compare the overdampled and critically damped case given the same initial conditions. Equivalency of both the initial velocity and position gives two equations.
Equating positions at $t = 0$ gives
$$ A_1 + A_2 = B_1 $$
and equating initial velocities:
$$ \mu_1 A_1 + \mu_2 A_2 = \omega_0 B_1 - B_2 $$
This means $A_2$ is not necessarily a free parameter. Plug in initial conditions (e.g. $x(0) = 1$ and $x'(0) = 0$) and then compare the behvior of the two system responses.
A: If your initial condition is such that $A_2$ is exactly zero, then indeed, in the absence of the second term in the rightmost object in (1), the first one would decrease faster than both terms rightmost object in (2).
But exactitude does not exist in the real world. Even if initially $A_2$ is as small as you can arrange it to be, it will never be exactly zero. And since both terms in (2) decrease exponentially faster than $e^{-\mu_2 t}$, however small a non-exactly zero value of $A_2$ is present in the very carefully prepared initial state you use, eventually the term that damps at the slowest rate, namely $A_2 e^{-\mu_2 t}$ will dominate.
Of course, if you prepare your initial state very carefully, so $A_2$ is extremely small, by the time $A_2 e^{-\mu_2 t}$ dominates the motion everything will be below the precision of your measuring apparatus. Then the experimental curve would show a faster decrease for the "heavy damping" up to the point where it is lost in the thickness of the ink of the  line.
I suppose that, for cases where $\gamma$ is just barely larger than $2\omega_0$ it could even be done experimentally. At the price of an enormous effort.
But who cares ?
A: There are good mathematical answers to this question. I'd like to give an intuitive one.
Consider the following situation: a sphere falling through a viscous fluid:

Equilibrium is when the sphere sits at the bottom without any movement.
Damping can be increased by making the fluid more viscous.
An underdamped situation is when the fluid's viscosity is too low. When we drop the sphere, it will bounce back and forth before settling down.
In a very overdamped situation, the sphere will take a long time to fall through the fluid to reach the bottom.
In the perfectly balanced situation (critical damping), the sphere falls as fast as it possibly could and there is no rebound. It just touches the bottom surface and sits there. This will be the fastest as compared to previous two cases.
A: What the OP asks for is indeed possible but only for very specific initial conditions.
There are a few things to discuss, the first being what does "returning to equilibrium faster" mean. First, we are trying to compare the behavior of two different systems (I will assume they have the same damping coefficient $\gamma$ but different resonant frequencies $\omega_{0,1}$, $\omega_{0,2}$), so it makes sense to only compare solutions to the two systems with the same initial conditions.
Second, unless the system is at rest with $x(t) = 0$ forever, the system is never actually in equilibrium and takes an infinite amount of time to reach equilibrium. One possible way to define "returning to equilibrium faster" would be that, given some fixed initial conditions $x_0, v_0$ , the solution $x_1(t)$ to system 1 approaches $0$ asymptotically faster than the solution $x_2(t)$ to system 2, both satisfying $x_1(0) = x_2(0) = x_0$ and $x_1'(0) = x_2'(0) = v_0$.
Another possible way to define it would be to compare the times $T_1, T_2$ it takes for $x_1(t), x_2(t)$ to get below a certain fixed value $M$, i.e. $T_1$ is the smallest $t \geq 0$ such that $\left|x_1(t)\right| < M$ for all $t > T_1$, and similarly for $T_2$. Then $x_1$ returns to equilibrium faster than $x_2$ if $T_1 < T_2$.
I will work with the first definition in terms of the asymptotic behavior.
In that case as you already mentioned, you are looking for initial conditions for which $A_2 = 0$. In that case, since $\mu_1 > \frac{\gamma}{2}$ and $\mu = \frac{\gamma}{2}$, the solution to system 1 would approach $0$ asymptotically faster than the solution to system 2.
This means that our solution in the overdamped system is $x_1(t) = A_1e^{-\mu_1t}$. Looking at $t = 0$, we have $x_1(0) = A_1$ and $x_1'(0) = -\mu_1 A_1$. Therefore, it is possible for a solution to system 1 to reach equilibrium faster than a solution to system 2 if the initial conditions satisfy the property that $x'(0) = -\mu_1 x(0)$. As $\mu_1$ is positive, and assuming we start from a positive value of $x$, then this just means that you need to give an initial velocity with just the right amount in the negative direction to achieve this. However, if we are starting at rest (zero initial velocity), it is impossible unless we are already at equilibrium.
One explicit numerical example for such a situation: let $\gamma = 4$ and let $\omega_0 = \sqrt{3}$ in the overdamped case and $\omega_0 = 2$ in the critically damped case. Let the initial conditions be $x(0) = 1$ and $x'(0) = -3$.
The differential equation in the overdamped case is $x''(t) + 4x'(t) + 3x(t) = 0$. The solution satisfying $x(0) = 1$, $x'(0) = -3$ is $x_1(t) = e^{-3t}$ as can be easily checked.
The differential equation in the critically damped case is $x''(t) + 4x'(t) + 4x(t) = 0$. The solution satisfying $x(0) = 1$, $x'(0) = -3$ is $x_2(t) = (1 - t)e^{-2t}$ as can also easily be checked.
And indeed $e^{-3t}$ returns to $0$ asymptotically faster than $(1 - t)e^{-2t}$ so this is an example of the phenomenon the OP is asking for.
As mentioned before, we had to pick the initial conditions just right to make this work. In practice there will never be an exact precision in the initial velocity that is given, so there will always be some component decaying at the slower rate. However, it may still be possible to make the magnitude of this component small enough for short times, and then we can use the second definition of "returning to equilibrium faster" to produce more realistic examples of the phenomenon.

Another thought: the previous discussion was based on the assumption that we keep the damping coefficient $\gamma$ the same and vary the resonant frequency $\omega_0$ to achieve overdamping or critical damping. However it might be more interesting to keep $\omega_0$ constant and vary $\gamma$ instead. So we have damping coefficients $\gamma_1 > \gamma_2 = 2\omega_0$ for the overdamped system and the critically damped system, respectively.
Then $\mu_1 = \frac{\gamma_1}{2} + \sqrt{\frac{\gamma_1^2}{4} - \omega_0^2}$, $\mu_2 = \frac{\gamma_1}{2} - \sqrt{\frac{\gamma_1^2}{4} - \omega_0^2}$, $\mu = \omega_0$
Since $\mu_1$ is a strictly increasing function of $\gamma_1$ and $\mu_1 = \omega_0$ when $\gamma_1 = 2\omega_0$, it is again true that $\mu_1 > \mu$. Similarly $\mu_2 < \mu$. So we can proceed exactly as in the previous discussion to find initial conditions for which the term involving $\mu_2$ is zero, and keep the term involving $\mu_1$ which decays faster than critically damped, for a fixed $\omega_0$.
A: 
to compare the heavy damping path and the critical damping path you have to adjust the initial conditions  you obtain
$$x_C(t)=\left( c_{{1}}+c_{{2}}t \right) {{\rm e}^{-\beta\,t}}$$
and
$$x_H(t)=w_1\,e^{\mu_1\,t}+w_2\,e^{\mu_2\,t}\\
w_1=\frac 12\,{\frac {c_{{1}}{\omega_{{0}}}^{2}-c_{{1}}{\beta}^{2}-\sqrt {-{
\omega_{{0}}}^{2}+{\beta}^{2}}c_{{2}}}{{\omega_{{0}}}^{2}-{\beta}^{2}}
}
\\
w_2=\frac 12\,{\frac {\sqrt {-{\omega_{{0}}}^{2}+{\beta}^{2}} \left( -\sqrt {-{
\omega_{{0}}}^{2}+{\beta}^{2}}c_{{1}}+c_{{2}} \right) }{{\omega_{{0}}}
^{2}-{\beta}^{2}}}
$$
$~\beta=\frac{\gamma}{2}$
hence:  critical damping return to equilibrium fastest  because the area under the
"critical path" $~\int_0^{T} x_C(t)\,dt~$ is smaller then the area under the "heavy path" $~\int_0^{T} x_H(t)\,dt~$

the initial conditions:
$$\mu_1=-\beta+\sqrt {-{\omega_{{0}}}^{2}+{\beta}^{2}}\\
\mu_2=-\beta-\sqrt {-{\omega_{{0}}}^{2}+{\beta}^{2}}$$
$$x_H(0)=x_C(0)\quad\Rightarrow \\w_{{1}}+w_{{2}}=c_{{1}}\\
\dot x_H(0)=\dot x_C(0)\quad\Rightarrow \\-w_{{1}}\beta+w_{{1}}\sqrt {-{\omega_{{0}}}^{2}+{\beta}^{2}}-w_{{2}}
\beta-w_{{2}}\sqrt {-{\omega_{{0}}}^{2}+{\beta}^{2}}=c_{{2}}-\beta\,c_
{{1}}$$
from here you obtain $~w_1,~w_2~ $ as a function of $c_1~,c_2,\omega_0~,\beta$
A: Critical damping is the condition which transitions between overshoot of the static equilibrium position (under damped) and an exponential decay to the static equilibrium position (over damped).
With less damping than critical damping the system will reach the static equilibrium position quicker than one which was critically damped but will overshoot and continue oscillating about the static equilibrium position with an exponentially decaying (never ending) amplitude.
With more damping than critical damping the system will always lag behind a critically damped system as it will lose mechanical energy faster and reach the static equilibrium position exponentially (never ending).
Critical damping means that the system will reach the static equilibrium position with no overshoot and with zero velocity and hence will be the fastest.
A: Simplest answer is that it is just a matter of jargon. Obviously, with very little damping the system may vibrate increasingly (unstable), or the vibration may dampen out after a long time.
With a lot of damping it will get to the stable position but it may take long, very long, or "forever".
So somewhere in between is the amount of damping where the system gets stable in the shortest time. That amount of damping will be called "critical damping".
