If I understand what he means he is saying that also $\psi$ and $e^{i \alpha} \psi$ are physically equivalent configurations and so must be counted once in the path integral too. Ok, but once we gauge fix we also take care of this ambiguity for free together with the well known one $A_\mu \to A_\mu +\partial_\mu \alpha$.
When he says that gauge fixing has to to with the invertibility of the operator $k^\mu k^\nu - k^2 \eta^{\mu\nu}$ he is referring to the kinetic term of the photon. Indeed if you Fourier transform $F_{\mu\nu}F^{\mu\nu}$ where $F_{\mu\nu} = \partial_\mu A_\nu - \partial_\nu A_\mu$ you find (after integrating by parts)
$$
S_{\mathrm{kin}} = \int d^4 k \,A_\mu(k^\mu k^\nu - k^2 \eta^{\mu\nu})A_\nu\,.
$$
So that's the operator you need to invert to find the propagator. It's the same as $\varphi (k^2 + m^2) \varphi$ leading to the propagator $1/(k^2+m^2)$. Since this operator has a null eigenvector, namely $k_\mu$, we cannot invert it.
But what is the meaning of this singularity? We know that the photon has $2$ degrees of freedom (the two transverse polarizations $\vec{E}$ and $\vec{B}$). However when we describe it as a Lorentz vector we implicitly allow for a longitudinal polarization too ($A_\mu \sim k_\mu \alpha$). You can immediately see that this mode doesn't have a kinetic term because $k_\mu$ kills the matrix, accordingly, its propagator is not defined.
So this singularity is just telling you that there are some modes without a kinetic term, so they are not dynamical. Which precisely means that you shouldn't sum over them in the path integral!
Let me try to be even more precise (or maybe more confusing). Every field has a certain number of polarizations, and so schematically it can be represented as follows:
$$
\Phi_A(x) = \sum_i^s\epsilon_A^{(i)} \,f_i(x)\,.
$$
the $f_i(x)$ are the modes and each of them satisfies a Klein-Gordon equation $(\square +m^2) \,f_i(x) = 0$. The $\epsilon_A^{(i)}$ are some polarizations fixed by symmetry. When I path integrate over $\Phi_A$ I actually path integrate over the $f_i$'s and each of them will have a source associated to it. Their kinetic term must have the form
$$
S_{\mathrm{kin}} = \int dx\,\sum_i c_i \,f_i (\square + m^2)\,f_i\,.
$$
where the $c_i$'s are constants. Redundant modes have no kinetic term and thus their $c_i$ is zero. Call $j_i$ the source for $f_i$. Then each path integral gives
$$
\int [d f_i]\,e^{- c_i \,f_i(\square + m^2)\,f_i + \int dx\,f_i\, j_i} = \exp\left(\frac 12 \int dx\, \frac{1}{c_i}j_i(x) (\square + m^2)^{-1} j_i(x)\right)\,.
$$
And the final result is the product of all these. You see that integrating over redundant configurations yields precisely the singularity that we find by inverting its kinetic term.
In the photon example we'd have three polarizations, $f_{1,2}$ being the transverse ones and $f_3$ being the longitudinal one, for which $c_3 = 0$.
