Suppose that I have a state $q$ that is constrained to satisfy $\mu(q)=0$. Assuming that $\nabla_q \mu(q)$ is a matrix of full-rank, I can use the Jacobian of the constraint function to define tangent spaces $\text{T}_q = \{\dot{q} : \nabla_q \mu(q)\dot{q} = 0\}$, the usual idea being that $\text{T}_q$ represents the space of velocities along coordinate curves that a curve passing through the point $q$ may exhibit while satisfying the constraint $\mu(q)=0$.
Beginning with a Lagrangian $\mathcal{L}(q, \dot{q}) = T(q, \dot{q}) - U(q) - \lambda^\top \mu(q)$ taking a Legendre transform reveals a Hamiltonian $H(q, p) = T(q, p) + U(q) + \lambda^\top \mu(q)$ where $p = \nabla_{\dot{q}} \mathcal{L}$ is the momentum. For the state variable $q$, its time derivative is $\dot{q} = \nabla_p H(q, p)$ and I can convince myself that the dynamics $\mathrm{d} q = \dot{q} ~\mathrm{d}t$ will satisfy the constraint by a Taylor expansion argument: $$ \mu(q_0 + \dot{q}~\mathrm{d}t) = \underbrace{\mu(q_0)}_{\text{assume = 0}} + \underbrace{\nabla_q \mu(q)^\top(\dot{q})}_{\text{tangent space constraint = 0}}\mathrm{d}t + \underbrace{O((\mathrm{d}t)^2)}_{\text{vanishes = 0}} = 0 $$ What I cannot show is that the dynamics of the momentum also satisfy its constraint. Namely, the momentum has to satisfy, at a point $q$, $$ J_q(p) = \nabla_q\mu(q)\nabla_pH(q, p) = 0 $$ and the dynamics for $p$ are $\mathrm{d}p = \dot{p}~\mathrm{d}t$ where $\dot{p} = -\nabla_qH(q, p)$. If I try Taylor expansion again, $$ J_q(p_0 + \dot{p}~\mathrm{d}t) = J_q(p_0) + (\nabla_q\mu(q)\nabla_p^2H(q, p_0))[\dot{p}] ~\mathrm{d}t + O((\mathrm{d}t)^2) $$ But it isn't clear to me that $\dot{p}\in\text{Ker}\{\nabla_q\mu(q)\nabla_p^2H(q, p_0)\}$.
