For example, if you roll a bowling ball straight down a lane, then run up beside it and kick it towards the gutter, you apply a force towards the gutter, but the ball doesn't go straight into the gutter but keep. Instead it keeps going down the lane, andbut picks up a little bit of diagonal motion as well.
If you throw the rock out horizontally, it will still fall, but it will keep moving out horizontally as it does so, and falls at an angle. (The angle isn't constant - the shape is a curve called a parabola, but that's relatively unimportant here.) The the force is straight down, but that force doesn't stop the rock from moving horizontally.