They say that gravity is technically not a real force and that it's caused by objects traveling a straight path through curved space, and that space becomes curved by mass, giving the illusion of a force of gravity.

That makes perfect sense for planetary orbits, but a lot less sense for the expression of gravity that we are the most familiar with in our day-to-day lives: "what goes up must come down."

Imagine that I hold a ball in my hand, several feet off the ground, with my fingers curled around it, and my hand is above the ball.  Then I open my fingers, releasing my grip on it, being very careful to not impart any momentum to the ball from my hand as I do so.

An object at rest remains at rest unless acted upon by an outside force.  If the ball is not moving (relative to my inertial reference frame), it has no path to travel that's any different from the Earth's path through space.  It should remain at rest, hanging there in the air.  And yet it falls, demonstrating that an outside force (gravity) did indeed act upon it.

How does curved space explain this?