But can we go the other way around and deduce the equation of motion as I described above? Perhaps by solving the differential equation
$$m \ddot{\mathbf r} = - \lVert \mathbf F \rVert \frac{\mathbf{r}}{\lVert \mathbf r \rVert}$$ where $||\mathbf F||$ is constant
This would not be the correct equation of motion.
You can just apply Newton's second law in Polar Coordinates:
$$\mathbf F=m\mathbf a=m(\ddot r-r\dot\theta^2)\,\hat r+m(r\ddot\theta+2\dot r\dot\theta)\,\hat\theta$$
For a force of constant magnitude always pointing towards the origin we have $\mathbf F=-F\,\hat r$, and so the equations of motion become
$$m\ddot r-mr\dot\theta^2+F=0$$ $$r\ddot\theta+2\dot r\dot\theta=0$$
which hold for any initial conditions.
In order to have uniform circular motion, we need
- $\dot r$, $\ddot r$, and $\ddot\theta$ to all be $0$ and,
- $r$ and $\dot \theta$ to be non-zero (they also need to be constant, but that follows from point 1).
This occurs when
$$F=mr\dot\theta^2$$ $$\dot r=0$$
So this shows that in order to have uniform circular motion we need for our initial conditions
- The force magnitude is equal to $mr(0)\cdot(\dot\theta(0))^2$
- $\dot r(0)=0$
If these two properties are not true of the initial conditions then you will not get uniform circular motion. You can determine what the motion will be from the general equations of motion we obtained above.