The point of potential energy is to say that the amount of KE in an object doesn't resemble all it's potential to do work(like a battery, or gasoline they both have ability to do work even though their momentum is 0).
This contradicts the concept of conservation if momentum, which says that the amount of potential to do work, is always there as speed and mass.
Here are some examples.
When you hold a ball above the ground it has Some PE. When you drop the ball, it loses PE and gains KE, increasing the ball's momentum. Clearly the momentum wasn't conserved.
When a driver hits the gas pedal in his car, it starts accelerating, turning PE of the fuel to KE of the car, increasing the car's momentum. Before he hit the gas pedal, the car had 0 momentum (relative to earth). The momentum wasn't conserved.
There are thousands of examples like those.
Where did my intuition go wrong?