Why is the electron self-energy gauge dependent? Let $\psi(x)$ be the field of the electron. Its Fourier transformed two-point function reads
$$
\langle\psi\bar\psi\rangle=\frac{1}{\not p-m-\Sigma(\not p)}.
$$
If we calculate $\Sigma(\not p)$, we observe that it depends on the gauge parameter $\xi$, which in principle is not a problem because $\Sigma(\not p)$ is not observable by itself.
But if we think of a gauge transformation as taking $\psi\to\mathrm e^{i\alpha(x)}\psi(x)$, then the two-point function should satisfy
$$
\langle\psi\bar\psi\rangle\to \langle\psi\mathrm e^{i\alpha(x)}\mathrm e^{-i\alpha(x)}\bar\psi\rangle=\langle\psi\bar\psi\rangle
$$
Therefore, one would naïvely expect $\Sigma(\not p)$ to be gauge invariant, and therefore it shouldn't depend on $\xi$. What is the solution to this contradiction? Why do our expectations fail?
 A: The propagator $S(p)$ is the Fourier transform of the two-point function
$S(x,y)=\langle\psi(x)\bar\psi(y)\rangle$,
$$
 S(p) = \int \frac{d^p}{(2\pi)^4} \, \exp(-ip\cdot(x-y)) \, S(x,y)\, . 
$$ 
Note that because of Lorentz invariance $S(x,y)$ does not depend on $x+y$. Clearly, the two-point function is non-local and not gauge invariant.
A: As an alternative to Thomas answer, we note that if we write the transformation law explicitly, we get
$$
\langle\psi(x)\bar\psi(y)\rangle\to \langle\psi(x)\mathrm e^{i\alpha({\color{red}x})}\mathrm e^{-i\alpha({\color{red}y})}\bar\psi(y)\rangle=\mathrm e^{i(\alpha(x)-\alpha(y))}\langle\psi(x)\bar\psi(y)\rangle
$$
We see that the two-point function fails to be gauge invariant because the fields are evaluated at different points and thus the local phases don't cancel off each other. This wasn't evident in the OP because I didn't write the space-time labels explicitly. Silly me.
A: The propagator - or any, arbitrary correlation function - depends strongly on the gauge of internal photons (the Ward identity deals with the variations of external photons' gauge).
This was first noted by Landau and Khalatnikov (and around the same time by Fradkin) who basically analyse the quantised version of the gauge transformation field called $\alpha(x)$ by OP:

*

*L. Landau, I. Khalatnikov, Sov. Phys. JETP2,69 (1956).

*E. S. Fradkin, Zh. Eksp. Teor. Fiz.29, 258261 (1955).

The treatment of $\alpha$ as a Stueckelberg type field is clearer in

*

*M. A. L. Capri, D. Fiorentini, M. S. Guimaraes, B. W. Mintz, L. F. Palhares, S. P. Sorella, Phys. Rev. D 94, 065009 (2016)

*T. De Meerleer, D. Dudal, S. P. Sorella, P. Dall'Olio, A. Bashir, Phys. Rev. D 97, 074017 (2018)

For the generalisation to arbitrary Green functions (involving simple products of the fermion field - see comments) see

*

*T De Meerleer, D. Dudal, S. P. Sorella, P. Dall'Olio, A. Bashir, Phys. Rev. D 101, 085005 (2020)

*N. Ahmadiniaz, J. P. Edwards, J. Nicasio, C. Schubert, arXiv:2012.10536 [hep-th]
