the object feels there is a radially outward force
No, it does not, and this is the whole reason behind the misconception.
A human may feel this way, but this is a thing that goes on in your head, and simply means that we are not wired in a way that we can really process the forces at play in a meaningful way on a biological level - mostly because it never happens in real life, so there is no selector for evolution to develop it.
Tie a rope to a large rock and twirl it around you (in free space with no planet close by, to avoid any confusion with its gravity). Cut the rope with one fell swoop. In this moment, no more forces act on the rock. The rock does not fly radially away from you; it flies away tangentially to the circle it just described. There was no real force pointing away from you.
You can easily test this in your garden, or from your armchair by watching a hammer throw in slow motion. Note how the athletes keep rotating a quarter turn after letting go, before following the flying object with their eyes.
While the rock was still bound, there acted exactly one force on it: the force of the rope pulling straight towards you; or from the perspective of the rock, towards its "bottom" (i.e. the centripetal force - from latin "pedes: on foot", like in "pedestrian"). While rotating around you, at each moment it had a speed tangential to its circle or orbit; and the centripetal force is the one that makes it "fall" towards you, or in more neutral terms, constantly changes its direction.
In the case of a solid rotating object, the rope is replaced by the electromagnetic forces that keep the particles together. In the case of an object standing on a planet, the rope is replaced by gravity. Oh, and a fun thought experiment: if someone would switch off gravity right now, on Earth, and a totally inelastic object were sitting on a completely inelastic floor (i.e., with no internal "spring" forces in the picture), and maybe in a local vacuum, it would seem to start flying parallel to the floor, not towards the sky.
The illusion of the centrifugal force is common to us and created by our brain; for example if you sit in a car and go fast around a corner, your head seems to move towards the side of the car. This is not the case though. Your head wants to go straight ahead; the side of your car is moving centripetally and thus pushing on your head if you relax your muscles enough that your head rests against the side.