I'm confused about how a contact term vanishes when proving the Ward identity, i.e. the spot immediately following equation 5.52 in Weigand's notes. Spelling out everything concretely, we consider a process with an external photon and some external fermions, giving an $S$-matrix element $$\langle f | i \rangle \sim \xi^\mu \int dx dx_1\ldots \, \partial_x^2 \not\partial_1 \ldots \langle 0 | T A_\mu(x) \psi(x_1)\ldots | 0 \rangle $$ and apply the Schwinger-Dyson equations to turn this into $$\langle f | i \rangle \sim \xi^\mu \int dx dx_1 \ldots \, \not\partial_1 \ldots \langle 0 | T j_\mu(x) \psi(x_1) \ldots | 0 \rangle $$ while throwing away a contact term, as explained here. Now we set the photon polarization $\xi^\mu = k^\mu$ and integrate by parts for $$\langle f | i \rangle \sim \int dx dx_1 \ldots\, \not\partial_1 \partial^\mu \ldots \langle 0 | T j_\mu(x) \psi(x_1) \ldots | 0 \rangle.$$ We now apply the Ward-Takahashi identity, generating more contact terms which are supposed to be zero; however, I don't understand why they are. Directly applying Ward-Takahashi gives a contact term for every external fermion, one of which has the form $$\langle f | i \rangle \sim \int dx dx_1 \ldots \, \not\partial_1 \langle 0 | T e\psi(x_1) \ldots | 0 \rangle \delta(x - x_1) \propto \int dx_1 \ldots \, \not\partial_1 \ldots \langle 0 | T \psi(x_1) \ldots| 0 \rangle.$$ Unlike the previous case, this seems to have precisely the right kind of pole structure; it's exactly what you would get if you applied LSZ.
What am I missing here? Why doesn't this contact term contribute?