Can auxiliary fields be thought of as Lagrange multipliers? In the BRST formalism of gauge theories, the Lautrup-Nakanishi field $B^a(x)$ appears as an auxiliary variable
$$\mathcal{L}_\text{BRST}=-\frac{1}{4}F_{\mu\nu}^a F^{a\,\mu\nu}+\frac{1}{2}\xi B^a B^a + B^a\partial_\mu A^{a\,\mu}+\partial_\mu\bar\eta^a(D^\mu\eta)^a,$$
and in the superfield formalism of SUSY, the field $F(x)$ also appears as an auxiliary variable:
$$\mathcal{L}_\text{SUSY}=\partial_\mu \phi\partial^\mu\phi+i\bar\psi^\dagger\bar\sigma^\mu\partial_\mu\psi+F^*F+\ldots\,.$$
It is very tempting to view $B^a(x)$ and $F$ as Lagrange multipliers since their equations of motion leads to constraints.  But, these variables do not enter into the Lagrangian linearly, like a conventional Lagrange multiplier.  Rather, they enter into the Lagrangian quadratically.
However, in Kugo and Ojima's paper Manifestly Covariant Canonical Formulation of the Yang-Mills Field Theores (1978), they refer to the $B^a(x)$ fields as the 'Lagrange Multiplier' fields (p.1882).
So my question is:  Can these auxiliary fields be viewed as Lagrange multipliers?  and in what ways do they behave differently/similar to the conventional Lagrange multipliers that enter into the function linearly?
 A: 1) OP writes (v1):

Can auxiliary fields be thought of as Lagrange multipliers?

No, not necessarily. Auxiliary fields usually mean non-propagating fields, and there may be other non-propagating fields, e.g. ghost fields and antighost fields. In case of so-called reducible gauge symmetries, one also has e.g. ghosts-for-ghost fields. Moreover, if one works in the Hamiltonian formalism, one has non-propagating momentum fields for all the above mentioned auxiliary fields. In fact, the converse statement is true: Lagrange multipliers are examples of auxiliary fields.
2) Strictly speaking according to the original definition, it is true that the Lagrange multipliers $\lambda^a$ should enter linearly (as opposed to e.g. quadratically) in the action,
$$S~=~\int \!d^4x ~{\cal L}, \qquad  {\cal L}~=~ \ldots + \lambda^a \chi_a+ \ldots,$$
where $\chi_a\approx 0$ are the conditions that we impose via the Lagrange multiplier method.
In Lagrangian gauge theories, the conditions $\chi_a$ are typically gauge-fixing conditions, and it turns out that when treated consistently$^1$, the gauge-invariant physical observables do not depend on the choice of gauge-fixing conditions $\chi_a$. 
This independence of the gauge-fixing conditions $\chi_a$ extends to
situations where the gauge-fixing conditions $\chi_a$ themselves depend e.g. linearly on the auxiliary $\lambda$ fields, so that the action depends quadratically on the $\lambda$'s. 
Many aspects of a gauge theory can be discussed before choosing particular gauge-fixing conditions, and in practice the auxiliary $\lambda$ fields are referred to as Lagrange multipliers anyway whether or not the gauge-fixing conditions $\chi_a$ themselves depend on $\lambda^a$. See also e.g. this Phys.SE answer.
For example, the gauge-fixed Yang-Mills action that OP mentions(v1) can precisely be viewed as a situation where the gauge-fixing conditions $\chi_a$ themselves depend linearly on the Lautrup-Nakanishi fields $\lambda$.
--
$^1$ Note that the Faddeev-Popov determinantal term also depends on the gauge-fixing condition $\chi_a$. For general gauge theories, a consistent treatment is given by the Batalin-Vilkovisky (BV) recipe, cf. e.g. this Phys.SE answer.
A: By definition, Lagrange multipliers are only coefficients that enter the extremized quantity (action) etc. linearly – and that multiply constraints. In some exceptional cases, an auxiliary field could enter in this way. However, they typically appear in a more complicated way and bilinear terms in the auxiliary fields are a rule rather than an exception. So strictly speaking, they're not Lagrange multipliers. But they are very similar. If no derivatives of these objects appear in the action, they're also "non-dynamical" (not involving time derivatives) and the variation with respect to them implies "non-dynamical" i.e. algebraic equations of motion.
Note that in the normal treatment of extremization, we introduce Lagrange multipliers because we want to extremize the quantity given the assumption that another quantity or other quantities are kept fixed. "Kept fixed" is translated as "conservation laws" into the physics jargon. However, in physics, we rarely consider conserved quantities that are conserved because the conservation law is explicitly written down as an independent constraint. Instead, in physics we usually discover conservation laws nontrivially – the conserved quantity has to be determined by a somewhat non-trivial procedure due to Emmy Noether out of a symmetry. In almost all physical theories, conservation laws are non-trivial consequences of some other, "more elementary" equations of physics.
