Below is the image depicting my confusion. In the top diagram, there is a horizontal rod with X marking the axis of rotation. Since the rod is horizontal, intuitively the rod will not rotate. This is indeed true since when I take the torque at the center of mass I get 0, as no other force is acting on the rod except gravity, which is 0m away from the axis.
However, the 2nd image depicts the same uniform rod, but tilted. Same as before, X is the axis located at the center of mass. Intuitively, I'd expect it to rotate towards to the horizontal state, and then probably swing back and forth. But taking the torque at CM still results in net torque of 0, as I can only identify the force of weight acting along the axis of the rod. This concludes that the rod should not rotate.
So my question is: Am I forgetting other forces present when I'm taking the torque and that's the reason why I missed out on why the 2nd rod would rotate, or is the intuition I have because of some real-life imperfections to the system that exists?