Not sure if I am posting this question in the correct community, as it relates primarily to reinforcement learning. Apologies early on if this is not so.

In reinforcement learning many algorithms exist for 'solving' the cart-pole problem; that of balancing a mass on the edge of a stick, connected to a cart on a hinge, which has 1 DoF. There is TD learning, Q-learning and many other on and off-policy methods. There is also the more recent, model-based policy search method PILCO.

What I am really wondering, I suppose, is more of a physics question: is there a need for active control? Why is it not possible to find the one point for the cart, which prevents the mass to move, even incrementally, left or right as it sits atop the pole? Why does it always 'fall'?

enter image description here

  • $\begingroup$ Your cart-pole is probably not operating in a perfect vacuum, and/or the pivot point is composed of more than one atom (not a mathematical point) and its temperature is above absolute zero. On the other hand, the pivot point is not perfectly friction-free. See unstable-equilibrium. $\endgroup$ – RedGrittyBrick Nov 25 '15 at 17:12
  • 1
    $\begingroup$ Personally I don't think this is the best community for this type of question, which seems very related to learning modes. The physics question in there is also not very clear, at least not to me. $\endgroup$ – Gert Nov 25 '15 at 17:39
  • $\begingroup$ I cross-posted it in robotics as well, as I was not sure where to put it... $\endgroup$ – Astrid Nov 25 '15 at 19:11
  • $\begingroup$ Really the answer to your (final question) is inertia. The stick has inertia. And if you move the cart the stick resists motion in that direction. This is why it 'falls back'. If you don't move the cart, and don't live in Southern California or Oklahoma (earthquake country) then you might be able to keep the stick from falling if there is enough friction. $\endgroup$ – docscience Dec 31 '15 at 1:55
  • $\begingroup$ ... but if the friction is negligible you definitely will need active control to maintain the vertical by countering the falling motions in the same manner you balance your broomstick. $\endgroup$ – docscience Dec 31 '15 at 1:57

The upright pole is in a position of unstable equilibrium. If the pole deviates from the vertical by an angle $\theta$ then the torque rotating the pole away from the vertical is:


$$ T = mg \frac{\ell}{2} \sin\theta $$

The moment of inertia of a pole about one end is $m\ell^2/3$, so the angular acceleration will be:

$$ \frac{d^2\theta}{dt^2} = \frac{3 g}{2\ell} \sin\theta $$

The point is that even the tinest deviation from the vertical, i.e. non-zero value of $\theta$, will make the pole accelerate farther from the vertical and it will eventually fall.

In the real world the pivot isn't a point and will have some friction, so in practice you probably could balance the pole. How easy it would be to balance would depend on exactly how the system had been made.

| cite | improve this answer | |

Not sure if I'm getting the question right, still let me answer the way I understood it. In this system you have two equilibrium points: the first is trivial (stick hanging down as in the left picture) [stable equilibrium], the second is as in your left picture with the stick "standing" [unstable equilibrium]. How do these two points differ? If you move your system a little bit, the left one will oscillate a bit and just go back to the starting position (hanging down). The right one will fall. Now, you can get the right position (as you can make a pen stand on a table) but the smallest force (wind, vibrations of the floor, ...) will make it fall. I guess it's not necessary to mention, that by moving the car you can "fight" against that power. So that's what you basically do! You should also think about the moment of inertia that arises because your stick is not massless (?). It should even be possible to move the car from pos a to pos b. How would you do that?

I'd say: Firstly drive "back". let the stick fall a bit, then "fight" the force in a way, that it will stand again and pos b. Here, if you want to understand it in a "deeper" way.

| cite | improve this answer | |
  • $\begingroup$ The stick standing up is indeed a non-stable equilibrium state and if you admit a little real-world friction then you can call it a metastable equilibrium. There are many (infinite?) control solutions to maintain the pole vertical by moving the cart, however fewer if you consider constraints on the control. Kwakernaak & Sivan specifically addresses such a solution in their book: Linear Optimal Control Systems, Wiley & Sons, 1972 pp 383-389 $\endgroup$ – docscience Dec 31 '15 at 1:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.