If you have a look at my answer to When objects fall along geodesic paths of curved space-time, why is there no force acting on them? this explains how on a curved surface two moving observers will appear to exprience a force pulling them together. However two stationary observers will feel no force. The force only becomes apparent when you move on the curved surface.
This is true in general relativity as well, but what is easily forgotten by newcomers to GR is that in GR we consider motion in spacetime, not just space. You are always moving in spacetime because you can't help moving in time. Your speed in spacetime is known as the four-velocity, and in fact the magnitude of the four-velocity (technically the norm) is always $c$. So you can't help moving through spacetime (at the speed of light!)
and when spacetime is curved this means you will experience gravitational forces.
You are probably familiar with Newton's first law of motion. This says that the acceleration of a body is zero unless a force acts on it. Newton's second law gives us the equation for the acceleration:
$$ \frac{d^2x}{dt^2} = \frac{F}{m} $$
The general relativity equivalent to this is called the geodesic equation:
$$ {d^2 x^\mu \over d\tau^2} = - \Gamma^\mu_{\alpha\beta} u^\alpha u^\beta \tag{1} $$
This is a lot more complicated than Newton's equation, but the similarity should be obvious. On the left we have an acceleration, and on the right we have the GR equivalent of a force. The objects $\Gamma^\mu_{\alpha\beta}$ are the Christoffel symbols and these tell us how much spacetime is curved. The quantity $u$ is the four-velocity.
Now let's consider the particular example you describe of releasing a ball. You say the ball is initially stationary. If it was stationary in spacetime, i.e. the four velocity $u = 0$, then the right hand side of equation (1) would always be zero and the acceleration would always be zero. So the ball wouldn't fall. But the four velocity isn't zero.
Suppose we use polar coordinates $(t, r, \theta, \phi)$ and write the four-velocity as $(u^t, u^r, u^\theta, u^\phi)$. If you're holding the ball stationary in space the spatial components of the four velocity are zero: $u^r = u^\theta = u^\phi = 0$. But you're still moving through time at (approximately) one second per second, so $u^t \ne 0$. If we use the geodesic equation (1) to calculate the radial acceleration we get:
$$ {d^2 r \over d\tau^2} = - \Gamma^r_{tt} u^t u^t $$
The Christoffel symbol $\Gamma^r_{tt}$ is fiendishly complicated to calculate so I'll do what we all do and just look it up:
$$ \Gamma^r_{tt} = \frac{GM}{c^2r^2}\left(1 - \frac{2GM}{c^2r}\right) $$
and our equation for the radial acceleration becomes:
$$ {d^2 r \over d\tau^2} = - \frac{GM}{c^2r^2}\left(1 - \frac{2GM}{c^2r}\right) u^t u^t \tag{2} $$
Now, I don't propose to go any further with this because the maths gets very complicated very quickly. However it should be obvious that the radial acceleration is non-zero and negative. That means the ball will accelerate inwards. Which is of course exactly what we observe. What is interesting is to consider what happens in the Newtonian limit, i.e. when GR effects are so small they can be ignored. In this limit we have:
If we feed these approximations into equation (2) we get:
$$ {d^2 r \over dt^2} = - \frac{GM}{c^2r^2}c^2 = - \frac{GM}{r^2} $$
and this is just Newton's law of gravity!