# Can the work-energy theorem be applied in this case?

I have tried to solve this problem by applying work energy theorem. The logic I applied is, the energy of the smaller bock dissipated by friction of the bigger block is equal to the work done on the bigger bock and is equal to the change in its energy. This gives me the following equations

Now I want to know where I went wrong as the answer I reached is wrong and also if we consider M>m (which is obvious) then the velocity is coming out to be an imaginary number which is impossible.

In the second pic the answer is a bit wrong as the $V_0$ will be squared.

You have made a sign error. The correct starting eqn is $$\frac{1} {2} m (v_0^2 - v^2) = \frac{1} {2} M v^2$$ The left is the energy lost by the small block, the right is the energy gained by the large block. Of course this assumes no energy is dissipated by friction, only transferred, which is unlikely.
This is easiest to see in the center-of-mass frame, where the momentum begins and ends at zero. If two objects come together and then stick together, then the kinetic energy must have been greater than zero before the collision, and zero afterwards as they both must be at rest in this frame of reference. The fact that they must have zero total momentum in this frame of reference means $$m_1 v_1 + m_2 v_2 = 0$$which can be rewritten as $v_2 = -(m_1/m_2) v_1$, their relative velocity is therefore $v_0 = v_1 - v_2 = \left(1 + \frac{m_1}{m_2}\right)v_1 = -\left(1 + \frac{m_2}{m_1}\right)v_2.$ If you try to actually add these fractions like we all learned in school you might see that there is an interesting mass $M = m_1 m_2/(m_1 + m_2)$ emerging here, with $v_0 = m_1~v_1/M = -m_2~v_2/M.$
The energy lost can therefore be derived in this center-of-mass frame as $$E_\text{loss} = \frac12 m_1 v_1^2 + \frac12 m_2 v_2^2 = \frac12 \left(\frac{M^2}{m_1} + \frac{M^2}{m_2}\right) v_0^2 = \frac12 M v_0^2,$$ where I have used the fact that $\frac1M = \frac1{m_1} + \frac1{m_2}.$
Differences in kinetic energy are reference-frame-independent even though the absolute value of kinetic energy is not, so your actual energy balance in the frame you're interested in needs to be (with your $M$ that I was calling $m_2$, not the $M$ I was using above)$$\frac12 m v_0^2 =\frac12 m v^2 + \frac 12 M v^2 + \frac 12 \frac{m M}{m + M} v_0^2.$$From this one is only a short step from finding $$\left(\frac{m}{m + M}\right)^2 v_0^2 = v^2$$and thus that $v = \pm v_0 m/(m + M),$ and it's "obvious" that we choose the + sign.
However, it is much easier to get to this result by, instead of using an energy balance, constructing the momentum balance, $$m v_0 = (M + m) v,$$which gives you that particular result directly without having to do any of this hard work.