If a mass M is at a point travelling at a given velocity (= speed and direction vector) then it has a given amount of energy (potential and kinetic energy summed) relative to being stationary* at the same point with respect to a given frame of reference. ie solely by knowing position and velocity vector the PE & KE are defined. (* or at some zero reference velocity).
This applies whether the mass is in a gravitational field or in a (local) gravity free field. (The energies will be different in the two cases but in each case velocity and position define the potential and kinetic energy for that case).
If the mass now leaves that point and "wanders about" in some manner and then returns to the same point with the same velocity vector (and with nothing else changed) then the energy will be identical to what it was before. ie just as PE & KE were determined solely by position and velocity vector previously they will again be now. If during the process of "wandering around" no energy was taken from or added to the mass by known and/or measurable mechanisms then, as the energy is unchanged, energy must have been conserved.
In a gravitational field or even in "free space" with no gravity, most things that we do to cause "wandering about" to happen, will require energy input or output from the mass. Usually, if we wish to "turn right"or slow or accelerate or "do a U turn" etc then we need to apply a force and to do so work will be done.
What Feynman is pointing out is that in this instance, although the velocity vector rotates through 360 degrees in direction (in the "orbital plane") and even though velocity increases to some maximum (aka perigee) and decreases to some minimum (aka apogee) and even though the mass travels in an elliptical path (which to our minds might suggest it is being acted on by forces which vary constantly in magnitude and direction), we find that when it passes through the original reference point it has identical velocity as originally, and it can do this time after time after time after ... forever (in the absence of the universe growing old etc) with exactly the same velocity at the same point every time. There is no "very small amount of energy lost or altered" - we know that in the absence of other influences a basic "2 body system" works just the same after 1 or 100 billion "orbits".
ie the mass starts with a given energy, undergoes changes in acceleration, velocity and position and returns to an exactly identical state as before with the same defined energy. So, energy is conserved in the process of completing an orbit.
The strangest thing about completing an "orbit" is, as Feynman notes, that we do not realise that it is strange.
As a bonus - it happens that for this to work a result drops out of the basic math which shows that attractive forces must be proportional to 1/(distance^N)
where N = 2.000000.... exactly.
ie the inverse square law must be EXACT.
Not 2.0000000000 ...1... or any other value.
This is another way of putting what Feynman put as
" it moves in such a manner that the square of the speed at any point minus
some constants divided by the radius at that point is always the same at
every point on the orbit"