Energy is the integral
$$W=\int_C \vec F\,d\vec s,$$
where $C$ is the trajectory, $d\vec s$ is infinitesimal displacement, and $\vec F$ is the force. If you don't know integral calculus, you can imagine this as the sum of small contributions of $F\Delta S$ along the trajectory.
Since you only briefly apply the force, it'll be nonzero only on a finite part of the trajectory $C$. After you stop applying the force, the integrand (i.e. the contributions $F\Delta S$) becomes zero for the remaining part of the trajectory, so further movement of the ball no longer contributes anything to the energy you calculate.
If your force is always in the same direction and is fixed in magnitude (while you're applying it), and you start with the ball at rest, then the energy will become
where $\Delta x$ is the length of the part of the trajectory where you were still applying the force.