In yang-mills theory , the constraint algebra closes to form a lie algebra. Even string theory has a constraint algebra which closes to form a lie algebra. I wish to know if there are other cases where the constraint algebra doesn't close. What does it physically mean for the constraint algebra to not close ?
In the Hamiltonian formulation, the non-closure of the constraint algebra is typically associated with second-class constraints and/or anomalies. See also my related Phys.SE answer here.