What breaks the symmetry in a Keplerian orbit? At first glance, one might think that the two body problem with inverse square force is a symmetric problem. The force (in the reduced mass frame of reference) depends only on the distance $r$, without an angle dependence.
However, most orbits are elliptical, hence something had to break the perfect symmetry. It cannot be angular momentum, since circular orbits, which are perfectly symmetric, are allowed. And in any case, angular momentum does not break 2D rotational symmetry.
So what breaks the 2D rotational symmetry in the two body problem with inverse square force? Is it the initial velocity?
 A: *

*In general if EOMs possess a symmetry, it is not necessarily true that a solution to the EOMs possesses this symmetry. See also this related Phys.SE post.


*It might make the reader more happy to know that the 3D Kepler problem is maximally superintegrable (i.e. has 5 independent integrals of motion), and via a coordinate transformation it is equivalent to a free particle on a 3-sphere $\mathbb{S}^3$, which has $SO(4)$ symmetry.
A: The eccentricity, $e$, of the orbit of the reduced mass, $\mu = m_1m_2/(m_1 + m_2)\,,$ is
$$
e = \sqrt{1 - \frac{L^2}{GMa\mu^2}}\,,
$$
where $a$ is the length of the semi-major axis of the ellipse, $M$ is the total mass of the system and $L$ is the magnitude of the total orbital angular momentum of the system.
The total angular momentum of the system is
$$
{\bf L} = m_1{\bf r}_1\times{\bf v}_1 + m_2{\bf r}_2\times{\bf v}_2\,.
$$
So it is the angular momentum that causes an uncircular orbit, i.e. an elliptical orbit with $e\ne 0$. So if the positions and velocities of the masses are such that
$$
L < \sqrt{GMa\mu^2}
$$
the orbit of the reduced mass will not be circular, but elliptical. This is what ``breaks the symmetry."
Orbits with
$$
0 < L < \sqrt{GMa\mu^2}
$$
will be elliptical.
${\bf Edit:}$ So yes, if you take given positions, the initial velocities will determine the eccentricity of the orbit. If these are such that
$$
L < \sqrt{GMa\mu^2}
$$
then the circular symmetry will be broken.
A: "Is it the initial velocity?"
Yes, if the initial velocity is not radial, it will break axial symmetry.
