What makes an orbit stable or unstable? I have an assignement, where I have a given central potential $V(x)=-\frac{K}{6r^6}$ and  object with an angular momentum $L$.
I've calculate the radius of a circular orbit, which I've done by solving
$$\frac{dV_{eff}}{dr}=0$$
which gave me $r=\sqrt[4]{\frac{mK}{L^2}}$
How to tell if this circular orbit is a stable or an unstable one? Thanks for your help!
 A: The other answers here are in the spirit of what you can do, but allow me to elaborate a little more.
To understand if the trajectory of the movement under a potential $V $ is stable or not you have to understand what this stability means.
The most simple example is the harmonic oscillation- $V=-{1 \over 2}kx^2 $.In Newtonian mechanics, for a point of stability,you have that $F(r_0)=0 $ where $r_0 $ is the point of stability. That is equal(if the $\nabla \times \bar F$ is zero) to the curve of the potential. So we have to find whether the $$\nabla V(r_0)=0 $$ gives us minimum values for the potential $V(r_0)$ or maximum values.  Then, in a diagram V=V(r) the maximum obviously gives you, for a little transposition dr,  the body moving away from an unstable point of stability. In the other hand, a minimum will give for the same transposition, a movement inside that like-a-well potential unless more energy is given to the body than the topical minimum has. Here's a picture:

It is useful and maybe fundamental to see this diagram in phase space(space of velocity and coordinate). Here:

The problem with this method is that if the scalar function $V(r)$ is a function of many variables(more than one) the mathematics require a lot of work.
So we use the method of disturbance(note: maybe my translation to English here is wrong). That means to take the force $F$ and write the Taylor expansion around a point(like mentioned in another answer) at distance e:
$$F(x_0+e)=F(x_0) + \left({dF \over dx}\right)|_{x=x_0}e + {1 \over 2}\left({d^2 F \over dx^2}|_{x=x_0}\right)e^2 +... =0$$. From here we take a linear differential equation whose solution is the behaviour of the particle
around $x_0$.
And next, if we have a Lagrangian $L=T-V $ where T the kinetic energy, with the general coordinates q in relation with the Cartesian are not a function of time, we have:
$$T=\sum_{i=1}^n \sum_{j=1}^n a_{ij}(q_1 ... q_n)\dot q_i \dot q_j $$ and $$V=V(q_1 ... q_n) $$ the equations of Lagrange are: $$\sum (q_1 .. q_n) \ddot q_i  + \sum \sum b_{ijk}(q_1 ... q_n)\dot q_i \dot q_j  + {\partial V \over \partial {q_k}}=0 $$ and k=0,1,2..n
For equilibrium solution the terms with derivatives over time are zero and so: $$\left.{ \partial V \over \partial {q_k}} \right|_{q_0} =0 $$.
For a little movement around the equilibrium point we can then prove:
$$L=\left(\sum_{i=1}^n m_{ij}\ddot χ_i + b_{ij}χ_i\right)=0 $$
Finally, who do we study the stability of circular orbits?
We have, for the equilibrium points: $$F(r)+{L^2 \over mr^3}=0 --> {dV(r) \over dr}=0 $$
For stable trajectory $V(r=r_0)''>0 $ where $V''$ is the second derivative over r of the potential. from here we can prove that:$$V''|_{r_0}=-F'|_{r_0} +{3 \over r_0}{L^2 \over mr{_0}^3}>0 -->{F'|_{r_0} \over F|_{r_0} + }+ {3 \over r_0 }>0$$. That's the condition for stable trajectories.
I know that's a lot, but I hope it helps.
A: One method for stability anayisis in a potential is to consider $r$ and $r+\epsilon$ where $\epsilon$ is a small number. If you can solve the dynamics analytically or run a numerical computer simulartion for your object with both $r$ and $r+\epsilon$ and observe with time if  $\epsilon$ gets smaller or larger that will give you a clue as to whether the orbit is stable or unstable (respectively). If it is unstable $\epsilon$ will grow.
A: Once you have the value 
$$dV_{eff}\over dr$$
This is the minus of the effective force i.e.:
$$F_{eff}=-\frac{dV_{eff}}{ dr}$$
If we differentiate this again we get:
$$\frac{dF_{eff}}{dr}=-\frac{d^2V_{eff}}{ dr^2}$$
If this is negative then at any small permutation around that radius will feel a force back to that radius and thus it is a stable orbit. If it is positive then the force will act to increase any small changes and thus it will be unstable. 
