Newton was the first to connect "falling motion" and the motion of "heavenly bodies". He said these two motions had the same origin: the gravitational force. But to explain how the Earth and other heavenly bodies do not fall, while the apple does, he needed to define how motion works and what quantities are involved. In what are now known as Newton's laws (because they seem to hold for all bodies) he said that bodies can and will move in a straight line with constant velocity if no forces are present, and forces acting together act like one equal to their sum whose result is changing the motion of the body possibly in both direction and magnitude, and that if a body exerts a force on another it will in turn feel an equal and opposite force. These elements, expressed mathematically allow to describe exactly the motions of the falling apple and the heavenly bodies.

But to answer your question maybe math is not necessary if we think of similar examples. While a force changes motion, it cannot avoid the inertia of the body, its tendency to "go as it was going". So a force applied sideways to the direction of motion will make it turn (slower or faster depending on how strong the force is compared to how fast the initial motion was). But by the time the turn happens, the body has moved forward a bit. Similarly when turning on a car, the friction of the street on the side of the wheel pushes it sideways, but does not make it turn in place, unless the force is strong enough. In such a way, the Earth is moving away from the Sun whose force acts sideways to the motion, but it can only make it turn a bit every time the Earth moves forward. Unlike the car, the Earth is moving in a frictionless space and loses almost no energy in its motion, so unlike the car, its forward motion is sustained for extremely long time.

Newton was the first to connect "falling motion" and the motion of "heavenly bodies". He said these two motions had the same origin: the gravitational force. But to explain how the Earth and other heavenly bodies do not fall, while the apple does, he needed to define how motion works and what quantities are involved. In what are now known as Newton's laws (because they seem to hold for all bodies) he said that bodies can and will move in a straight line with constant velocity if no forces are present, and the effect of forces acting simultaneously is such as if only one force was acting, equal to their sum. Whatever the force, its effect is a change in the motion of the body, possibly in both direction and magnitude. Finally he stated that if a body exerts a force upon another, it will in turn be subjected to an equal and opposite force. These elements, expressed mathematically, allow to describe exactly the motions of the falling apple and the heavenly bodies under their respective conditions.

But to answer your question maybe math is not necessary if we think of similar examples. While a force changes motion, it cannot avoid the inertia of the body, its tendency to "go as it was going". So a force applied sideways to the direction of motion will make it turn (slower or faster depending on how strong the force is compared to how fast the initial motion was). But by the time the turn happens, the body has moved forward a bit. Similarly when turning on a car, the friction of the street on the side of the wheel pushes it sideways, but does not make it turn in place, unless the force is strong enough. In such a way, the Earth is moving away from the Sun whose force acts sideways to the motion, but it can only make it turn a bit every time the Earth moves forward. Unlike the car, the Earth is moving in a frictionless space and loses almost no energy in its motion, so unlike the car, its forward motion is sustained for extremely long time.