Your analysis is correct up to the point when the two speeds are equal. This is also the point when the relative velocity is zero. But at this point the two balls are deformed (elastically). The forces acting on the two balls during approach is an elastic force. However small the deformation is, it is there. So what happens now is that indeed the two bodies start to fall apart (the distance between them increases) and they start to expand back to the original shape. The interaction force starts to decrease (does not drop to zero instantly). The work done during this second part of the process is the same as during the first part so the kinetic energies and velocities change by another on half. This is the difference between perfect elastic and non-elastic collisions. In the later case the two bodies do not expand back to the same shape (or not at all) so the exchange of velocities is not complete.
It may help if imagine a spring between the two bodies. When the velocities are equal, the spring is still compressed. Actually it has maximum compression. So obviously the force exerted by the spring is not zero as it starts to expand.
Note:A nice example is considering a rubber ball bouncing off a ground(suppose perfectly elastically), in that situation there is also a point where the relative velocity with the ground is zero(when the ball is momentarily at rest with the ground) but still the ball rises up due to elastic forces as it gains kinetic energy due to the energy stored through the deformation it had initially endured.
