Gravity governs projectile motion, so it is still accelerating the object downwards. The velocity at the time of exit is that it had at some point in the circle, which, instantaneously, was tangent to the circle at that point. That is the direction it moves off in, but gravity will accelerate it downwards.
In circular motion the velocity and acceleration are linked by a formula, derivable by twice differentiating the position vector to get acceleration. The centripetal acceleration is that component of acceleration perpendicular to the velocity, which (is the only one that) changes the velocity's direction. Gravity provides exclusively this kind of force at the top of the circle, when the velocity is horizontal, but as the velocity changes due to gravitational acceleration, or at some other point in the circle, the gravitational force has a smaller and smaller component perpendicular to the velocity.
When the particle attains a vertical velocity gravity has no component perpendicular to this and cannot accelerate it to turn any more; alternatively, the object might reach land first. The acceleration is thus sufficient to make a parabolic arc, but not circular motion.
So it is because gravity does not act perpendicular to velocity at every point, just downwards, that it cannot create circular motion.