I'd like to add a bit of mathiness to what Qmechanic says. In the first constrained optimization problem with $L(q) = V(q) + \lambda C(q)$, you are attempting to determine the minimum value attained by the function $L$ which is presumably a function on some subset of Euclidean space $\mathbb R^n$ (or perhaps a function on some neighborhood of a real, smooth manifold which pretty much amounts to the same thing). So you want to determine the point $q = (q^1, \dots, q^n)$, where each $q^i$ is a real number, at which the function $L$ attains a minimum value.
In the case of physical Lagrangians, it is true that the Lagrangian is still the same type of function (a function on "configuration space" in this case which is locally just $\mathbb R^{2n}$), but we don't really care to determine a point in configuration space at which the Lagrangian attains a minimum value. Instead, we consider parameterized paths $q(t)$ on configuration space and attempt to determine which paths minimize (actually we only really care to find critical paths which are not necessarily maxima or minima) the action functional $S[q]$ which is a function that takes a path on configuration space, and returns a real number. As Qmechanic writes, the action functional that we use in classical mechanics is defined as
$$
S[q] = \int dt\, L(q(t), \dot q(t), t)
$$
Note that the symbol $q$ which we are here using to label the argument of the action functional $S$ denotes a path as opposed to a point in configuration space (it's a rather common abuse of notation in physics to use $q$ to denote both points in configuration space and paths). Notice that the Lagrangian is, in this case, simply an intermediate function with which we write the action functional. In particular, we don't so much care to minimize the Lagrangian itself.
Cheers!