The tree level 4-fermion amplitude you get when you collapse the propagator to a point is a $J_\mu^+ J^{\mu ~-}$ contact term, the 4-Fermi interaction of the 30s.
With the advent of functional integrals to the SM, the relevant part of the SM action contributing to this charged current amp is $$ {\cal L}_{eff}= m^2_W W_{\mu}^+ W^{\mu ~-}+ \frac{g}{\sqrt 2}( W_{\mu}^+ J^{\mu -} +W^{\mu ~-} J_\mu^+) +O(p^2/M_W^2). $$ Complete the complex square, $$ {\cal L}_{eff} = m^2_W \left (W_{\mu}^+ +\frac{g}{\sqrt {2} m_W^2} J^{\mu~+}\right) \left(W_{\mu}^- +\frac{g}{\sqrt {2} m_W^2}J^{\mu~-} \right ) -\frac{g^2}{2m_W^2} J_{\mu}^{+} J^{\mu~-} . $$ Now observe the first term represents a shift in the definition of the Ws; integrated over spacetime and stuck in the exponent of the functional integral, it amounts to two Gaussians, when you resolve it into the original "new", shifted W1, W2; functional integration of these Gaussians w.r.t. the shifted Ws leaves no trace of the Ws in this low energy part of the path integral. They have been "integrated out", as per your question.
The sole usable residue of their presence, is the "constant" (as far as W degrees of freedom are concerned) second term, the current-current interaction, $-\frac{2G_F}{\sqrt 2} J_{\mu}^{+} J^{\mu~-} $, where one defined $G_F\sqrt 2 \equiv g^2/4m_w^2$. Note you would get the very same answer from the merely algebraic equations of motion of ${\cal L}_{eff}$, namely $W_\mu^{\pm}=-g J_\mu^{\pm}/\sqrt{2} m_w^2$; using these to eliminate the Ws would result in the same current-current residual interaction.
(By the way, in 1933, this was essentially the first application of QFT: its linchpin feature of creation and annihilation of fermion species.)
A very analogous procedure, naturally, obtains for the neutral current amplitudes involving Z exchange.