Why the extra term $\frac{1}{2}(\partial_{\rho}A^{\rho})^2$ in the photon Lagrangian? In my quantum field theory class we have been told to use this Lagrangian for the photon field
$$\mathcal{L}=-\frac{1}{4}F_{\alpha\beta}F^{\alpha\beta}
-\frac{1}{2}(\partial_{\rho}A^{\rho})^2.$$
but the reasons have been kept quite obscure. Why do we do this? We don't we stick with the good old Maxwell Lagrangian? Doesn't this extra term change things? and if not (as I suppose it is) why not?
 A: The extra term, in general
$$\mathcal{L}=-\frac{1}{4}F_{\alpha\beta}F^{\alpha\beta}-\frac{1}{2 \xi}(\partial_{\rho}A^{\rho})^2 $$
is called gauge fixing term. This term is needed in order to be able to quantize the field $A_\mu$. Without this extra term the photon propagator is ill defined
$$D^{\mu\nu}={-i\over k^2+i0}\left(g^{\mu\nu}\,+\,(\xi-1){k^\mu k^\nu\over k^2+i0}\right)$$ for $\xi\rightarrow \infty$, which corresponds to working in the unitary gauge, i.e. the Lagrangian without the extra term.
Furthermore, the canonical commutator relations could not be fulfilled, because $\Pi_0=0$, if we use the Lagrangian without the extra term. If you want to read about these matters have a look at the Gupta-Bleuler procedure. An alternative approach would be to fix the gauge first and the quantize. Nevertheless, a covariant quantization procedure, as the Gupte-Bleuler formalism is preferred.
A: The Lagrangian provided is Maxwell's Lagrangian, supplemented by a gauge fixing term:
$$\mathcal{L}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu} - \frac{1}{2}(\partial_\mu A^\mu)^2$$
The equations of motion are,
$$\partial_\mu F^{\mu\nu} + \partial^\nu (\partial_\mu A^\mu) = \partial_\mu \partial^\mu A^\nu = 0$$
Instead of making a gauge fixing procedure a posteriori, a term is added to the Lagrangian so that the gauge fixing condition arises naturally from the equations of motion. The reason we are at liberty to impose such a condition is because of the gauge symmetry under a transformation,
$$A_\mu \to A'_\mu=A_\mu + \partial_\mu \epsilon(x)$$
for an arbitrary function $\epsilon(x)$. If we choose to identity $A_\mu$ with $A'_\mu$, then we can always take a 4-potential and make it satisfy $\partial_\mu A^\mu = 0$ (Lorenz gauge) providing we solve,
$$-\partial_\mu \partial^\mu\epsilon(x)=\underbrace{\partial_\mu A^\mu}_{\mathrm{original \, \, potential}}$$
for the appropriate $\epsilon(x)$. When quantizing electromagnetism, one will find the commutation relation,
$$[a^\lambda_p,a^{\lambda' \dagger}_q] = -\eta^{\lambda \lambda'}(2\pi)^3 \delta^{(3)}(p-q)$$
for the creation and annihilation operators. This is disturbing because it means for timelike polarization states with $\lambda = 0$, states will have negative norm, i.e.
$$\langle p,0 | q, 0 \rangle = -(2\pi)^3\delta^{(3)}(p-q) < 0$$
To resolve the issue, we demand that physical states $|\Psi \rangle$ satisfy an adaptation of the Lorenz gauge condition due to Gupta and Bleuler, namely,
$$\partial^\mu A^{+}_{\mu} |\Psi\rangle = 0$$
where $A^+$ is the decomposed field, containing only the expansion with the $a$ term, rather than $a^\dagger$ term.
