@DivMits's answer is on the right track, but I think more detail is needed, and a final conclusion needs to be stated.
Before we get started, we should first address an issue raised in the comments around some confusion about what type of friction the coefficient $\mu$ is associated with. To get around this issue we could be as general as possible and specify coefficients of static and kinetic friction for each block (i.e. $\mu_{s,1}$, $\mu_{k,1}$, $\mu_{s,2}$, $\mu_{k,2}$). However, based on the wording of the problem, solutions, and comments, I think we can get away with assuming that $m_1$ and $m_2$ are made of different materials such that we arrive at the contrived case that $\mu$ is the coefficient of kinetic friction for $m_1$ as well as the coefficient of static friction for $m_2$. If this assumption is too specific we can change it later, or the more general case can be left to others to explore on their own. I think it would be a fun exercise.
So, let's first start off by redefining out goal. If we want $m_2$ to move, then we need the spring force to be equal to the maximum static friction force. Therefore, assuming the spring starts at its unstretched length, we need to move $m_1$ a distance $X$ such that
$kX=\mu m_2g$. So, instead of asking "What is the minimum force is needed to move $m_2$?" we can now shift the focus entirely onto $m_1$ and ask:
"What is the minimum force needed to move $m_1$ a distance $X=\mu m_2g/k$ ?
Case 1
Based on your solution, it looks like we are applying a constant force $F$ the entire time. So, let's think about the scenario of the minimum force. Well... we would want to move $m_1$ a distance $X$ such that it is at rest when it reaches $X$. If $m_1$ was moving at this point then we would have "tried too hard", so to speak. This is when your solution applies the conservation of energy, saying that we put in energy $FX$ and it went into potential energy $\frac12kX^2$ and work done by kinetic friction on $m_1$ equal to $\mu m_1gX$. Since $m_1$ is at rest, there is no kinetic energy to consider. Therefore, you get the equation given (after dividing by $X$):
$$F=\frac12kX+\mu m_1g=\frac12\mu m_2g+\mu m_1g$$
This is how the problem should be solved. Once you start trying to look at forces acting on $m_1$ after it has moved a distance $X$ you are now shifting your focus and answering different questions, as discussed below.
Case 2
Through writing, thinking, re-writing, re-thinking, and so on, I have determined your issue here with case 2. There is no unique way to balance the forces acting on $m_1$, and there is no reason to suspect that the equilibrium of $m_1$ coincides with when $m_1$ has moved a distance $X$. What you have done in your equation you give is one of two things:
- Assumed that $\mu m_1g$ is a kinetic friction force. Then you are assuming that $m_1$ is still in motion when it moves a distance of $X$. However, this means that you are applying a greater force than what we previously found (as you have shown in your work). The problem has not been correctly solved.
- Assumed that $\mu_s m_1g$ is a static friction force acting in the same direction as the spring force (Notice I have defined a new $\mu_s$ coefficient, since it is obvious that your book intends $\mu$ to refer to kinetic friction for $m_1$). This is where things get more complicated, because this assumes that the applied force is just large enough so that $F-kX=\mu_sm_1g$. In other words, it is a very specific scenario where we are requiring the maximum static friction force to be acting on $m_1$.
Since point 1 is invalid, let's think about point 2. You have changed your question and your scenario. Instead of just looking at what the minimum force is to get $m_1$ to travel a distance $X$, you are asking "What force do I need to apply to $m_1$ so that when it has moved a distance $X$ we are right at the maximum static friction force acting on $m_1$?" In other words, instead of trying to find the minimum force to move $m_2$, you are trying to find a different force that is not be equal to the minimum force you wanted to find. We know they cannot be equal, since if we set our actual minimum force equal to the new force in question, we end up with:
$$\frac12\mu m_2g+\mu m_1g=\mu m_2g+\mu_sm_1g$$
or
$$(\mu_s-\mu)m_1=-\frac12\mu m_2$$
This requires $\mu_s<\mu$, which is not possible in the simple model of friction we are using. Therefore both points 1 and 2 are not the correct way to tackle this problem.
In summary, case 1 is the right way to tackle the problem. Your case 2 adds unnecessary complications that ends up changing the system and goal of the problem. There is no reason to assume that when $m_1$ ends up at rest after moving a distance $X$ that the forces acting on $m_1$ should balance out. This is discussed below (mostly because I spent time thinking about it. Read ahead to see how much trouble we get into in trying to look at equilibrium on mass 1).
But can we still answer the question in just thinking about an equilibrium of forces? Based on the above discussion we cannot consider a minimum force when $m_1$ is in equilibrium with a kinetic friction force. Therefore, we will consider a general static friction force acting on $m_1$ when it is at rest after moving a distance $X$ such that $|F_s|=|F-kX|\leq\mu_sm_1g$. The magnitude and direction of the static friction force is determined by the relative strengths of $F$ and $kX$. Therefore, we have three cases to consider:
Case 3A: $F=kX$
This is the case where once $m_1$ has moved a distance $X$ it has no static friction force acting on it. Therefore, $F=\mu m_2g$. Can this be the answer? It turns out it can be if we have just the right system. From case 1 we know that the minimum force required is given by $F=\frac12\mu m_2g+\mu m_1g$. Therefore, setting these equal we get:
$$\frac12\mu m_2g+\mu m_1g=\mu m_2 g$$
or
$$2m_1=m_2$$
If the mass of block 2 is twice the mass of block 1, and if we supply a force exactly equal to $\mu m_2g$ to $m_1$, then $m_1$ will move a distance of $X$, come to rest, and then stay at this position (since the kinetic and static friction forces will be $0$. Therefore, if $2m_1=m_2$ and you naively apply balance of forces that $F=kX=\mu m_2g$ then you will technically be correct! If this condition is not met then you would be wrong for one of two reasons
- $2m_1<m_2$ : This means the kinetic friction force is weaker than before (or the static friction on $m_2$ is stronger). This means our $F=kX$ force will overshoot and try to bring $m_1$ past $X$. This means we have not found the minimum force.
- $2m_1>m_2$ : This means the kinetic friction force is stronger than before (or the static friction on $m_2$ is weaker). This means our $F=kX$ force will undershoot and $m_1$ will not even get to move a distance $X$ before coming to rest. This means we have not found the minimum force.
Therefore, your attempt of case 2 can be valid in the case where $2m_1=m_2$.
Case 3B & 3C: $F>kX$ or $F<kX$
Since $F\neq kX$ we must have a static friction force of magnitude $|F_s|=|F-kX|<\mu_s m_1g$ acting on $m_1$ once it travels a distance of $X$. Then the equilibrium of the forces on $m_1$ requires:
$$F\pm F_s=kX$$
where we have a positive sign on $F_s$ if $F<kX$ and a negative sign if $F>kX$.
Unfortunately this is as far as we can go here. We cannot determine what $F$ is without knowing what the static friction force is. (Of course we could determine what $F_s$ is given our solution in case 1, but that would no longer be solving the problem using balance of forces).
This mess of an answer just goes to show how when you try to shove in assumptions (equilibrium of forces acting on $m_1$) that are not valid how we end up on wild goose chases to make those assumptions work ("I am right if $2m_1=m_2$! My assumption works!")
