Why doesn't orbital body keep going faster and faster? If we consider the change in velocity during an infinitesimal interval of an orbit:

where body B is orbiting body A, we can see that the magnitude of the resultant vector (the green arrow) is greater than the magnitude of the original tangential velocity. Why doesn't this magnitude keep increasing indefinitely?
As I understand elliptical orbits, they speed up and slow down, but according to the diagram, I would expect them to keep speeding up monotonically.
(The answers to the duplicate question do not answer my question).
 A: Any body travelling with increasing velocity increases its kinetic energy $KE$. Since in your system:
$$KE+PE=\text{constant}$$
where $PE$ is potential energy. Therefore increase in $KE$ results into decrease in distance between the objects (so as increase $PE$).
Note: $KE$ is always positive. $PE$ can be positive or negative. $PE$ is negative for bound systems.
A: Your vector sum drawing is incorrect. Considering the simple case of a perfectly circular orbit, there is a tangential velocity vector and a perpendicular (inward) acceleration vector. Strictly speaking, you can't add them because they are different quantities (acceleration vs velocity). 
At some point in the orbit, the object has an initial velocity V in some direction we'll call X, and an inward acceleration in the perpendicular direction we'll call Y. One quarter orbit later, the acceleration vector has accelerated the object from zero to V in the Y direction, but it has also rotated (still pointing inward), decelerating the object from V to zero in the X direction. At this point, there is no more Y component to the acceleration.
If you break down the acceleration and velocity vectors into cartesian components, you find that each describes a sinusoid, peaking at some point, crossing zero one quarter cycle later, reaching an inverse peak one quarter cycle after that, crossing zero again after another quarter cycle, and returning to the initial peak at one full cycle. If you add up the area under the curves, you get zero - the negative exactly equals the positive, so there is no net change, so no net increase in velocity.
A: Since you still seem puzzled I'll try a different tactic here:
You're showing the tangential velocity (A) and the radial acceleration (B) and adding them to get the green arrow.  What you're missing is that this occurs in a gravity field.  As the initial path climbs away from the object it's orbiting it loses velocity.  This shows up as a third arrow pointing opposite of A and is of exactly the right amount to keep things balanced and your object peacefully in orbit rather than flying to infinity.
A: After reading the answers from other people, I see that you are still confused, so I thougth i coult take my chances on clearing up your confusion.
It seems that your confusion is why does the orbiting body not accelerate to infinite velocities (or crashes to body A) eventually, and the origin of your confusion is in your assumption that the resulting velocity is always greater than the previous one, which is not correct.
Let me start with a small example for visualization.
Asume that the A body is fixed and the body B to moves in a straigth line that doesn't cross the A body (For now let's ignore the change in direction of the velocity).
Now as the body B approaches body A, its velocity will increase until it reaches the point in which it's at minimum distance from body A, at this point the body B still has great velocity so it continues to move past this point, but also past this point the force generated between both bodies, will start to deaccelerate (reduce its speed). At some point the velocity will be 0, and the body B will go back to the possition of minimum distance.
If we leave this system alone, body B will oscillate periodically, never reaching maximum velocity, never crashing to A.
You would agree with me that such system does not resemble any physical system, however it should help you develop the necesary intuition.
I hope you are familiar with a Uniform Circular motion, but suffice to say that for any movement the acceleration that is perpendicular to the movement only changes the direction of movement.
Now we have the tools to tackle the full orbital motion.
At the closes point of the orbit, the acceleration is perpendicular to the movement and will not change the magnitude of the velocity, but even if it did, and even if it would increase the velocity as in your picture, you have that the velocity is large enough that the B body will get further away of the A body as it was earlier, but in such scenario, the force will start to de-accelerate the B mass, reducing it's velocity in the same fashion as it did in the system where the movement was constrained to a straigth line.
The B body will continue to get further and further away, while describing a curve given by the continuous pull in the radial direction up to the point in which it will not move further away and start going back to A repeating the same movement continously
I hope this clarifies your confusion.
A: You have to consider the limit of infinitesimally short time, in which the (vertical on the paper) component of velocity is infinitely short, and thus also the angle changes for an infinitesimal amount. In this limit, the correction to the length is quadratic in the time step and vanishes exactly in the physical limit of continuous time. Pythagoras:
$$v_2=\sqrt{v^2+(adt)^2}\approx v + \frac{a^2}{2v}dt^2 +\cdots$$
A: Because the direction of the velocity changes. The velocity will start to point less and less 'towards' point A and when the distance between A and B is the smallest, the velocity will make a right angle with the radius, which means acceleration vector also makes a right angle with the velocity. At this point the radial component of the speed is zero and the total speed is the highest. After this point the acceleration vector will point slightly away from the speed vector and its length will only decrease until it reaches the highest point again. 
I have made this image to help you understand with the velocity (red) and radial velocity vector (blue) drawn. Keep in mind that when the radial velocity is decreasing but still pointing towards A, the total velocity is still increasing.

A: This question points out the importance of symplecticity in physics.
In an orbital simulation, suppose one simply advances state via
$$\begin{align}
\boldsymbol x(t+\!\Delta t) &= \boldsymbol x(t) + \boldsymbol v(t)\, \Delta t \tag 1 \\
\boldsymbol v(t+\Delta t) &= \boldsymbol v(t) + \boldsymbol a(t)\, \Delta t
\end{align}$$
where $\Delta t$ is a finite (non-infinitesimal) quantity and $\boldsymbol a(t)$ is calculated via Newton's law of gravitation. This makes the orbiting body spiral outward and gain speed. This is what is vexing @TylerDurden.
This is outward spiraling is clearest when one starts with an object in a circular orbit. The initial step is along the tangent, so away from the circular orbit. The speed increases as well; the change in velocity is orthogonal to the initial velocity. Something is clearly amiss.
What's amiss is the discretization performed above, as suggested by simple numerical integration theory. Any second order degree differential equation can be converted to a first order differential equation by making the first derivative (velocity in this case) a part of the state and then applying numerical integration techniques for solving first order ODEs to the resultant differential equation. The simplest numerical solution to solving a first order initial value problem is to advance state via $\boldsymbol s(t+\Delta t) =   \boldsymbol s(t) + \Delta t \, d \boldsymbol s(t)/dt$. This is the Euler method, and it results in equations (1) above when applied to an orbiting body.
The problem is that this discretization isn't symplectic (i.e., it violates the conservation laws). Another way to look at is that this approach ignores geometry. (The conservation laws are "geometry.") There are other non-symplectic techniques such as canonical Runge Kutta integration that make an orbiting body spiral inward.
The issue at hand is that converting a second order differential equation to a first order differential equation and then using first order initial value techniques to numerically solve the ODE comes at a cost, and that cost is tossing geometry out the window. What's needed are techniques that do not toss geometry out the window. A very simple approach is to apply equations (1) in a slightly different order:
$$\begin{align}
\boldsymbol v(t+\!\Delta t) &= \boldsymbol v(t) + \boldsymbol a(t)\, \Delta t \tag 2 \\
\boldsymbol x(t+\!\Delta t) &= \boldsymbol x(t) + \boldsymbol v(t+\!\Delta t)\, \Delta t
\end{align}$$
This is the symplectic Euler method. Notice how the velocity and position calculations are now braided. This is one of the meanings of "symplectic."
If you work out the math with regard to applying equations (2) to gravitation, you'll find that this alternate formulation of the Euler method explicitly obey's Kepler's second law, that a line drawn from the Sun to a planet sweeps out equal areas in equal times. This is geometry! Kepler's second law is of course a special instance of the conservation of angular momentum. The conservation laws and geometry are closely coupled.
A: First, calculus isn't just really small steps: I can show you limit processes that disagree with any really small step based solution, like making a staircase with smaller and smaller treads.  The total "tread plus rise" size remains 2k, while the limit is a line with length sqrt(2)k.
However, almost all of the parts of calculus that work in predicting physics actually work with really small steps based systems.  So instead of delving directly into calculus, I'll start with your steps.
Lets examine what happens as we make the step size smaller.
Our orbiting body is at 1 AU.  It is orbiting the sun, which weighs ~ 2 10^30 kg.  It moves at ~ 29,870 m/s.  Gravitational acceleration is ~0.0060 m/s^2.  It takes about a year to orbit.
Over a timestep of t, the speed is:
$$\sqrt{(29870 \frac{m}{s})^2 + (0.0060 \frac{m}{s^2} * t)^2 }$$
or
$$29870 \frac{m}{s} * \sqrt{1 + (2.01 * 10^{-7} \frac{1}{s} * t)^2 }$$
by dividing out by the current velocity.  The part that makes the orbiting body go faster is the part under the square root: $\sqrt{1 + (2.01 * 10^{-7} \frac{1}{s} * t)^2 }$ -- when it is greater than 1, the velocity after the time step is greater.
What happens when t is ridiculously small?  Well, one way to figure this out is to take the Taylor Series1
$$\sqrt{1 + (2.01 * 10^{-7} \frac{1}{s} * t)^2 }$$
let $x=(2.01 * 10^{-7} \frac{1}{s} * t)^2$
$$=1+\frac{x}{2}-\frac{x^2}{8}+...$$
where we can guarantee that the prefix of this series is off from the "real answer" at infinity by the value of the next element in the series.
So $1$ is an approximation f the answer that is wrong by less than $\frac{x}{2}$.
Let us plug in Planck time for $t$, or ~$5 * 10^{-44}s$.
We get that the velocity of the orbiting body is its original velocity, plus at most 1 part in 10^100.
Suppose this body was orbiting for the current lifetime of the universe.  Then the amount of speed up we might detect is about 1 part in 10^60.
Basically, if the universe is continuous, then in the limit there is no additional speed.  If it is discrete on a really small scale, then the amount of additional velocity generated by such time steps would be undetectible at time scales we can probe, and possibly lost to "rounding" caused by space and time both being discrete.
If the universe is space and time quantized, the scale we expect it is at the Planck scale, far below what we can currently experiment with.  And we can show that our continuous calculus model of the universe generates an model of the universe that is close enough that we cannot distinguish between them using our current observational abilities.

1 the astute will notice I slipped in Calculus here.  Yes, I'm cheating.
