Can a physical system have initial velocity zero, but non-zero constant everywhere else? Does there exist a particular physical system where the initial velocity can be zero, but non-zero constant everywhere else?
 A: If the velocity is constant the force is zero. So we want a system which produces a very high force in an infinitesimally short time at $t=0$ and then drops back to zero. The simplest system that I can come up with right now is an ideal$^*$ billiard ball at rest which is hit by another billiard ball at $t=0$. It will have velocity zero before $t=0$ and then a constant velocity afterwards
$*$ with ideal I mean in the limit of an infinitely hard ball. If the ball is not infinitely hard the force will exist for a finite amount of time (the ball will act like a spring).
A: Exactly as you're putting it, no. Such a velocity would have a discontinuity, which means infinite acceleration, which implies infinite force.
It is possible, on the other hand, to consider that a force applies over a short duration $\tau$, in which case velocity would vary rapidly while remaining continuous:
$$dp=F\,dt
\quad\Rightarrow\quad
\Delta p=\int_0^\tau F\,dt
\quad\Rightarrow\quad
\Delta v=\int_0^\tau\frac{F}{m}\,dt$$
If initial velocity is zero, then velocity at $t=\tau$ is:
$$v(\tau)=\int_0^\tau\frac{F}{m}\,dt$$
If the force vanishes after $\tau$, then velocity will remain constant and equals to $v(\tau)$ after that.
