Total energy of orbiting body less than $-GMm/2a$?

After studying about effective potentials, I understand that circular orbits occur when the total energy of the orbiting body is a minimum, elliptical orbits when the total energy is between this value and zero, and so on so forth. My question is, what happens when the total energy is less than the energy of a circular orbit? For example, a spaceship in circular motion applying its brakes. My intuition tells me it will spiral towards the body it is gravitating about, and I understand that the eccentricity of the orbit will become imaginary, but I don't know if this intuition is correct, and I haven't found any literature online to confirm/debunk my understanding.

Edit: I can understand that the effective potential changes when the speed of the orbiting body changes, since the angular momentum changes. My confusion lies with the total energy. So, I understand that total energy comprises the gravitational potential energy and the kinetic energy. In a circular orbit, this total energy is a minimum of $-GMm/2r_0$. When the body applies brakes and slows down, at that instant, the gravitational potential energy remains constant and the kinetic energy decreases. What happens afterwards is what my brain cannot comprehend.

• You can't have an energy less than the minimum. – Javier Jul 1 '17 at 7:56
• I mean like for example, a rocket in circular motion suddenly reduces its speed from rockets or something, wouldn't the total energy be less than what it had during the circular orbit? – user107224 Jul 1 '17 at 8:03
• When you're using the effective potential, the "kinetic energy" is actually only the part associated with radial movement. And don't forget that the effective potential depends on the angular momentum, so "minimum energy" also implicitly depends on it; if you brake, you change the shape of the effective potential. In no case does an object spiral inwards. – Javier Jul 1 '17 at 8:06
• I see, but it still doesn't help me clear my doubts about the total energy. Does that mean when you brake, it is still possible for the total energy to be greater than the minimum? – user107224 Jul 1 '17 at 8:59
• Even after your edit, it isn't clear at all to me what is confounding your brain. It's true that, for a given angular momentum $L$, the circular orbit is the orbit of minimum total energy. If (somehow) you reduce the speed (in the direction of motion) of the body, $L$ necessarily decreases and so the body is no longer in a minimum energy orbit even though the body's total energy is reduced. Thus, it is now in an elliptical orbit. If the body were to stop, the body would be in a $L = 0$ radial orbit. For $L = 0$, there is no minimum for the effective potential. – Alfred Centauri Jul 1 '17 at 13:33

After studying about effective potentials, I understand that circular orbits occur when the total energy of the orbiting body is a minimum, elliptical orbits when the total energy is between this value and zero, and so on so forth.

You've missed a subtlety in the concept of effective potential.

The effective potential of a two body central force system with an inverse square law force is $$U_\text{eff}(r) = \frac{l^2}{2\mu r^2} + U(r) = \frac{l^2}{2\mu r^2} - \frac k r \tag{1}$$ where $r$ is the radius, $l$ is the angular momentum of the system, $\mu$ is the reduced mass of the system, and $U(r)=-\frac k r$ is the central force potential.

Differentiating equation (1) with respect to $r$ and treating $l$ as a constant results in the effective potential reaching a minimum at $r=l^2/\mu k$, where $U_\text{eff} = -\frac12 \frac{\mu k^2}{l^2} = -\frac12\frac k r$. The highlighted text is the subtlety you are missing.

Effective potential can obviously be less than this; simply substitute $l=0$ and you'll get an effective potential of $-\frac k r$. Note that this is effective potential, which is not the same as total mechanical energy. The total mechanical energy of the system is $\frac12 \mu v^2 - \frac k r$. Setting the velocity to zero yields a total mechanical energy of $-\frac k r$, the same result obtained by setting $l=0$ in the effective potential.

I am not sure I understand what you mean by an effective potential but I think your question about the trajectory of an object at a radius with a kinetic energy lesser than that of the object at the same radius in a circular orbit can be answered anyway.

If the mass of the planet is $M$ and you are at a radius $R$ then the trajectories change according to the following rules:

$v=0 \equiv \text{Straight-line}$

$0<v<\sqrt{\dfrac{GM}{R}} \equiv \text{Ellipse with the initial position as the aphelion}$

$v=\sqrt{\dfrac{GM}{R}} \equiv \text{Circle}$

$\sqrt{\dfrac{GM}{R}} <v<\sqrt{\dfrac{2GM}{R}} \equiv \text{Ellipse with the initial position as the perihelion}$

I think you understand the rest of the story. So, the trajectory below the circular orbit energy is also an elliptical orbit but with the initial position of the projectile becoming the aphelion. Notice that for a spiral motion, the planet would have to continuously radiate some energy away - so you could have straight-away discarded the spiral trajectory.