One way to look at this is to understand that the system will tend to move to a point of lower potential energy.
If we roll a ball on a surface, we expect it to end up at the bottom of a valley, not the top of a hill, because that is the lowest potential energy. Even if it starts at the top of a hill, we don't assume symmetry is a reason for it not to fall at all. That instead some minor (but unmodeled) forces are sufficient to nudge it in one direction or another.
In a rotating system, potential energy is higher when the mass is toward the center rather than when the mass is radially distant from the center. Mass near the center that can move will lose potential energy by moving away.
When the balls are "light" compared to the bottle, then it's not a big deal. Both balls can move to (the same) edge of the system and be in a nice local minimum state.
When the balls are massive, this can't happen because the center of mass must remain near the balls. If you had a fixed axis at the center of the tube, they could move to one end. But when it is freely rotating, the axis is through the center of mass, which is influenced significantly by the location of the balls.
So when both massive balls are together, they are necessarily near the center of rotation. The only way for them to move to a lower potential energy state is to separate.
The other way to look at this is that if both balls are on one side of the spinning axis, they will tend to stay there. (It would take energy to move one up to the axis to then go past). The larger and more massive the balls (with respect to the bottle), the closer the axis must be to them. For the case of tennis balls in a light plastic tube, you can't arrange for both balls to be on the same side of the axis. So that situation is unstable. For two ping-pong balls in a heavy bottle, you can, so that configuration is (meta)stable.