Lagrange multiplier and constraint force The Lagrangian with Lagrange multiplier in the form
$$L= T- V + \lambda f(q, \dot{q},t).$$
But there are different ways of writing the constraint $f = 0$.
Will that lead to different EOMs?
Let me give an example:
A pendulum with mass $m$ and length $\ell$.
We can use let
$$I=\int_{t_0}^{t_1}\left[\frac{1}{2}m(\dot{x}^2+\dot{y}^2)-mgy-\lambda\left(\sqrt{x^2+y^2}-\ell\right)\right]dt,$$
or
$$I=\int_{t_0}^{t_1}\left[\frac{1}{2}m(\dot{x}^2+\dot{y}^2)-mgy-\lambda\left((x^2+y^2-\ell^2)^2\right)\right]dt.$$
In the first case, we have
$$m\ddot{x}=-\lambda\frac{x}{\ell}, \qquad  m\ddot{y}=-mg-\lambda\frac{y}{\ell},$$
which are the correct EOM.
But for the second case, we have
$$m\ddot{x}=0,\qquad  m\ddot{y}=-mg.$$
 A: Let there be given a (configuration) manifold $M$. Often in physics one assumes that a constraint function $\chi$ obeys the following regularity conditions: 


*

*$\chi: \Omega\subseteq M \to \mathbb{R}$ is defined in an open neighborhood $\Omega$ of the constrained submanifold $C\subset M$;

*$\chi$ is (sufficiently$^1$ many times) differentiable in $\Omega$;

*The gradient $\vec{\nabla} \chi$ is non-vanishing on the constrained submanifold $C\subset M$.
Here it is implicitly understood that $\chi$ vanishes on the constrained submanifold $C\subset M$, i.e. 
$$C\cap \Omega ~=~\chi^{-1}(\{0\})~:=~\{x\in\Omega \mid \chi(x)=0\}.$$ 
[Also we imagine that the full constrained submanifold $C\subset M$ is covered by a family $(\Omega_{\alpha})_{\alpha\in I}$ of open neighborhoods, each with a corresponding constrained function $\chi_{\alpha}: \Omega_{\alpha}\subseteq M \to \mathbb{R}$, and such that the constraint functions $\chi_{\alpha}$ and $\chi_{\beta}$ are compatible in neighborhood overlaps $\Omega_{\alpha}\cap \Omega_{\beta}$.] Since there (locally) is only one constraint, the constrained submanifold will be a hypersurface, i.e. of codimension 1.  [More generally, there could be more than one constraint: Then the above regularity conditions should be modified accordingly. See e.g. Ref. 1 for details.]
The above regularity conditions are strictly speaking not always necessary, but greatly simplify the general theory of constrained systems. E.g. in cases where one would like to use the inverse function theorem, the implicit function theorem, or reparametrize $\chi\to\chi^{\prime}$ the constraints. [The rank condition (3.) can be tied to the non-vanishing of the Jacobian $J$ in the inverse function theorem.]
Quantum mechanically, reparametrizations of constraints may induce a Faddeev-Popov-like determinantal factor in the path integral.
Example 1a: OP's 1st example (v1)
$$\tag{1a} \chi(x,y)~=~x^2+y^2-\ell^2$$
would fail condition 3 if $\ell=0$. If $\ell=0$, then $C=\{(0,0)\}\subset M=\mathbb{R}^2$ is just the origin, which has codimension 2. On the other hand, the $\chi$-constraint satisfies the regularity conditions 1-3 if $\ell>0$.
Example 1b: OP's 1st example (v3) 
$$\tag{1b} \chi(x,y)~=~\sqrt{x^2+y^2}-\ell$$
is not differentiable at the origin $(x,y)=(0,0)$, and hence would fail condition 2 if $\ell=0$. On the other hand, the $\chi$-constraint satisfies the regularity conditions 1-3 if $\ell>0$.
Example 2a: Assume $\ell>0$. OP's 2nd example (v1) 
$$\tag{2a} \chi(x,y)~=~\sqrt{x^2+y^2-\ell^2}$$
would fail condition 1 and 2. The square root is not well-defined on one side of the constrained submanifold $C$. 
Example 2b: Assume $\ell>0$. OP's 2nd example (v3) 
$$\tag{2b} \chi(x,y)~=~(x^2+y^2-\ell^2)^2$$ 
would fail condition 3 since the gradient $\vec{\nabla} \chi$ vanishes on the constrained submanifold $C$. 
References:


*

*M. Henneaux and C. Teitelboim, Quantization of Gauge Systems, 1994; Subsection 1.1.2.


--
$^1$ Exactly how many times differentiable depends on application.
A: In general it is OK to do this as long as the constraint is holonomic. Any (sufficiently) differentiable function that vanishes on the submanifold of interest, is defined in an open neighbourhood of the latter, and whose gradient does not vanish there, will work well.
The reason for that is that any other such function $ h $ can be written in terms of the original function $ f $ as $$h=fg, $$ where $ g$ does not vanish on the constrained submanifold. This can change the magnitude of the gradient of $ h $ but not its direction:
$$\nabla h =g \nabla f + f\nabla g=g\nabla f. $$
More intuitively, the gradients of $ f $ and $ h $ must both be orthogonal to their shared contour, the constrained submanifold, and must therefore be linearly dependent.
In general, the corresponding equations of motion will be different, but they will have the same solutions. The Lagrange multiplier, including its time dependence, must obviously change.
Note, on the other hand, that this works as stated only for holonomic constraints; it should work for anholonomic ones but I can't quite see the proof. Finally, your second example, including a square root, is not defined on an open neighbourhood of the circle and is therefore not valid.
