I am assuming that we are not looking at a situation like a mass oscillating on a vertical spring. It seems like we are pulling on a mass with a spring so for this answer I will assume that the constant force was built up to slowly so that no oscillations are initiated. If we wanted to include ordinations then thinking about when $k\neq\infty$ would need to be approached differently.
$k$ is qualitatively the "stiffness" of the spring. The larger $k$ is the stiffer the spring is. So as $k$ goes to $\infty$, you basically just have a rigid body attached to the mass. Therefore, the force you apply to the spring is $F$, and the force the mass experiences is also $F$ (assuming a massless "spring").
Another thing to notice is that $U=\frac12kx^2$ is not a "transfer of energy". It just tells you how much energy is stored in a spring that is stretched a distance $x$ from equilibrium. It doesn't tell you anything about how much energy the mass gains.
Let's be more exact here. Let's assume a massless spring that we are applying our constant force $F$ to. Since the total mass of the system is $m$, the acceleration is $a=\frac Fm$. This means that a force of $F$ also acts on the mass, and so by Newton's third law the spring actually has a force of $F$ acting on both sides. Notice how none of this depends on the spring constant.
Your energy argument comes from misunderstanding the equation of elastic potential energy, as explained above. We already know that the spring exerts a force $F$ on the mass, and so the work done by the spring on the mass over a displacement $x$ is just $W=Fx$, which also does not depend on the spring constant.
The reason $k$ it's irrelevant is because for a constant applied force, the spring will achieve some equilibrium applied force that is independent of the mechanism from which this force arises. This same exact analysis and reasoning can be performed if we were instead using a massless string instead of a spring, or if we glued a block to the mass in question and pulled on that instead.