From Newton's laws, we know that an object in motion will travel in a straight line unless acted upon an external force. On the merry-go-round, you are constantly changing direction; i.e., your direction is always tangential to the center of rotation. If you draw a velocity vector at different times on the merry-go-round you will see this clearly. Thus the moment you release the ball it continues in a straight line, with its initial velocity, tangential to the center of rotation at the moment of release.
There is something called "rotational inertia" more commonly referred to the "moment of inertia", however, it is not as you describe. It a measurement of the inertia of a rotating object--that is the resistance to changes in angular acceleration (increasing or decreasing rotation speed). You can think of it this way, a very light piece of material (say foam) will be easy to spin, however, a large block of granite will be harder to spin. When I say easy and harder, I really mean easy/harder to accelerate into a spinning motion. Similarly, trying to stop the granite while spinning will be harder as it has a higher moment of inertia (because of density in this case). This is the principle behind flywheels, the storage of energy in a rotating member.
Even looking at these cases, the individual particles (atoms) all experience centripetal acceleration towards the center because the atoms are connected. If they were not connected the atoms would fly away in a straight line just like the ball does. In your example.
See this for example: high speed disintegration of a compact disk.
So we can say that Newton's Law describes the fact that all particles want to travel in a straight line. Rotation is possible because we exert a force on a particle to change its direction, but the particle itself wants to continue on a straight line path.