If two identical masses are somehow "released" into deep space (that is, they're subject to no other gravitation forces but their own, and are initially at rest to each other). What decides whether they collide or orbit each other? I'm imagining for example two 1 kg masses initially at rest, say 10 m apart. What happens next?
Due to the law of conservation of angular momentum they will only orbit if there is angular momentum in the initial conditions. If they start at rest, there is none and so they will collide.
Since we're supposed to provide links and whatnot, here's the wikipedia entry.
Part of the difficulty in answering this question, however, is that there are many different kinds of orbits and the details of the trajectories depends on the relevant masses and linear and angular momentums.
Assume: no visible stars, CMB, you on Earth and the Sun. You have a watch that is calibrated by the solar day. To simplify lets assume that the N-S direction of the rotation is normal to the plane of a circular orbit, if it is the case.
With a Foucault pendulum and assuming that exist invisible distant stars to avoid a Machian problem interpretation, you can say:
At each noon you can measure the angular diameter of the Sun. If it stays constant then you are orbiting, else you have to wait to decide.
If you dont have instruments the option is 'wait and see'. If the two bodies have angular momentum > 0 then you are orbiting. If a collision happened then it is = 0. How do you define an adequate referential in this situation? It has to be a referential centered in the Earth but you have no way to 'anchor' one axis except in the line Sun-Earth. With this referential you can not discriminate the motion except using the apparent diameter.
We have a solid bar ended with two masses as in this picture:
Aother problem that I address in my answer is the definition of a practical reference frame.
You can draw 3 axis not anchored in any objects and say it forms a reference? No, except if you consider the existence of an 'absolute observer' outside the problem. This is the case with you, and me, because we are exterior to the problem configuration and we can easily 'frame' the problem.
At last: an observer synchronously gravitating the rotating and featureless Earth above the day/night separation line will not sense any motion or force. To him the universe is static.
Thats why I used the Foucault pendulum and the CMB reference frame.
If the masses are at initially at rest wrt to the CMB frame (and one wrt to the other), then they will collide.
Note: I do not understand how the others answers are trying to analyse motion and forces without defining a reference frame.
EDIT add end
with no forces perpendicular to the vector connecting them, they will just collide on this line, at a distance proportional to the ratio of their masses.
The only way to avoid this situation is start with a different tangential velocity.
the collision of objects depends on the mass of each and its velocity vector relative to each other.