I want to find the Feynman propagator for the so called $R_\zeta$ gauge fixed electromagnetic field. The lagrangian density is given by: \begin{align} L = -\frac{1}{4}F_{\mu\nu}F^{\mu\nu}-\frac{1}{2\zeta}(\partial_\mu A^\mu)^2. \end{align} The equations of motion are therefore \begin{align} \partial_\mu F^{\mu\nu}+\frac{1}{\zeta}\partial_\mu\partial_\lambda A^\lambda g^{\mu\nu} = 0 \end{align} which can be formulated as the operator equation \begin{align} 0 = [g^{\nu\lambda}\square-(1-\frac{1}{\zeta})\partial^\nu\partial^\lambda]A_\lambda =:O^{\nu\lambda}A_\lambda \end{align}
The feynman propagator $D_{\nu\rho}(x-y)$ is defined as the greens function \begin{align} O^{\mu\nu}D_{\nu\rho}(x-y)=-ig^\nu_\rho\delta(x-y) \end{align} I now used a fourier ansatz to obtain \begin{align} -k^2g^{\mu\nu}\tilde{D}_{\mu\rho}+(1-\frac{1}{\zeta})k^\mu k^\nu\tilde{D}_{\nu\rho}+ig^\mu_\rho=0 \end{align} which can be solved for the fourier transformed propagator \begin{align} \tilde{D}_{\nu\rho}(k)=\frac{ig_{\nu\rho}}{k^2(3+\frac{1}{\zeta})} \end{align}
I am slightly confused by a hint at the exercise, whichs says that I should use the ansatz \begin{align} \tilde{D}_{\nu\rho} = A(k^2)k_\nu k_\rho+B(k^2)g_{\nu\rho} \end{align} and I didn't use this ansatz at all. Of course for $A=0$, $B=\frac{i}{k^2(3+\frac{1}{\zeta})}$ I get exactly what I found for $\tilde{D}_{\nu\rho}$ but I cant figure out what the author wants me to do with this information. Furthermore I found this post, where it is mentioned, that the Propagator is \begin{align} \tilde{D}_{\mu\nu} = -\frac{1}{k^2+i\epsilon}[g_{\mu\nu}-(1-\zeta)\frac{k_\mu k_\nu}{k^2}] \end{align} which does not agree with what I got. (They might use a different convention $\zeta \rightarrow \frac{1}{\zeta}$ here)