Constraints are handled in Lagranian mechanics through either of two approaches:
1) The constraint equation is used to reduce the degrees of freedom of the system. For example, if a particle is constrained to the surface of a sphere, then the Lagrangian can be written entirely in terms of two generalized coordinates and their associated momenta (typically, one chooses the polar and azimuthal angles of spherical coordinates to be the generalized coordinates). One cannot find the constraint forces this way--they are tacitly ignored as a result of reducing the complexity of the problem.
2) Alternatively, one adds a Lagrange multiplier as an extra degree of freedom. If one has a Lagrangian $L$ and constraint function $f(q_i, t)$ that is constrained to some value $C$ and is a function of the generalized coordinates and time, then one modifies the Lagrangian as follows:
$$L \mapsto L' = L + \lambda [f(q_i, t) - C]$$
Applying the Euler-Lagrange equations to this new Lagrangian $L'$ reproduces all the relevant dynamics of the system. In particular, it allows you to solve for the Lagrange multiplier $\lambda$. Doing so makes it possible to find the constraint forces, which are
$$F_i = \lambda \frac{\partial f}{\partial q_i}$$
It's important to note that such a constraint function $f$ must be a function of only time and the generalized coordinates. The term for this kind of constraint is holonomic. Problems with non-holonomic constraints cannot be treated in this way.