I apply a force. It seems natural that the ball should move forward.
Assuming no friction:
Sure , if you apply a force on the ball from inside, it's centre of mass will move forward slightly.

But the person experiences an equal and opposite force and it's centre of mass will move backwards such that the centre of mass of the combined Man + ball system will not move.
The force you apply is an internal force only if you consider the Man + ball as your system. If you take the ball as being a separate system, then the force you apply is an external force. Whether a force is internal or external depends on what you take as your system.
In this case, the ball and man will keep rotating about their combined centre of mass in opposite sense since the angular momentum has to be conserved.
Assuming friction is present:

Assume that initially, the COM of the man and the ball were along the same vertical line. As the man moves forward to push the ball, a net torque is produced about the COM of the ball and fricton acts in the forward direction so that finally pure rolling occurs ( assuming friction is sufficient) and the ball + man system moves forward as long as he applies force.