So you've seen that $\frac{1}{2} m \dot{\vec{x}}^2+ mgz$ doesn't describe everything. Now, you'll learn about how to go from a formula almost like this to equations of motion later, but let's ignore that for a bit. Let's say there's a potential $U(\vec{x})$ which is zero on the circle, and off the circle is equal to a constant $k$ times the shortest distance to the circle, squared.
This is like a rigid pendulum. Imagine $k$ getting larger and larger. Due to conservation of energy, the point in question is constrained to only move within a certain distance of the desired curve. As $k$ gets larger and larger, this distance of constraint shrinks. As $k\to \infty$, the distance shrinks to $0$.
Now in this idealization when the particle is on the path, $U(x)$ is - in this limiting sense - zero on the circle and infinite everywhere else. There are mathematical proofs that this works (or discussions of it) in Arnold, Mathematical Methods of Classical Mechanics.
The physical intuition behind this is that you can have a lot of force without any energy. If you compress water as hard as you can, when released you won't notice anything. If you compress air as hard as you can, when released there can be a big explosion.