A Gaussian (free) field theory is described by a quadratic action of the field, e.g. $S=\int\psi^\dagger K\psi$ (or $S=\frac{1}{2}\int\phi^\intercal K\phi$ for real fields). Usually one just need to diagonalize the action kernel $K$, then each eigen vector corresponds to a mode/polarization of the field. For example, $$S=\int \psi^\dagger(-i\partial_\tau+H)\psi\stackrel{\text{diag}}=\sum_n\psi_n^\dagger(-\omega+E_n)\psi_n,$$ where each $n$ labels a mode of the field $\psi$, and the dispersion relation (or the energy spectrum) is given by setting the eigen value to zero, such as $(-\omega+E_n)=0$ and hence $\omega=E_n$.
However when I tried to apply this approach to a gauge theory, I got some trouble. For instance, consider the Maxwell theory (with a Euclidian metric in 4 dimension), $$S=\int\frac{1}{4}F^2=\int\frac{1}{2}A^\mu\Pi_{\mu\nu}A^\nu,$$ where $\Pi_{\mu\nu}=k^2\delta_{\mu\nu}-k_\mu k_\nu$ is the action kernel, and $k_\mu=i\partial_\mu$ is the energy-momentum vector. $\Pi$ is a $4\times 4$ matrix which can be diagonalized. There will be one zero mode, corresponding to the gauge transformation of the gauge field (as can be seen from its eigen vector $A_\mu\sim\partial_\mu \phi$), which should not be counted as a physical mode. So far so good. But there are still three (degenerated) non-zero modes, with the same eigen value $k^2$. At this point, I would tend to conclude that there should be three physical modes, all degenerated on the dispersion relation $k^2=0$. But in fact, photon only have two transverse modes. My question is what is wrong with the mode counting? Shouldn't the longitudinal mode already excluded as the zero mode (gauge mode), but why we are still left with three eigen modes in $\Pi$?
If we perform a pseudo-inverse of $\Pi$, the photon propagator should be $$-(\Pi^{-1})_{\mu\nu}=\frac{1}{k^2}\left(\delta_{\mu\nu}-\frac{k_\mu k_\nu}{k^2}\right),$$ which also has three poles along the dispersion $k^2=0$. If each pole corresponds to a physical mode, then there will be three photon modes, which is still in contradiction with the fact that photon only have two transverse modes. How to do the mode counting correctly?