In the particle exchange picture, the particles are emitted in all directions and only the ones going from P in the direction of E that hit E are intercepted and have an effect. The other particles interfere themselves out of existence, as there is no on-shell state they can enter while conserving energy, or else return to P, giving the self-energy modification to P's mass. In fact, most return to P, since the self-energy is divergent, while only a small fraction make it to E by comparison.
This process is virtual, so that it is defined by temporary intermediate states which only can stick around until their phase randomizes them away. For the case of a classical force, you need to use particles that go every which way, forward and backward in time.
Consider two classical objects interacting with a (free) quantum field according to this Lagrangian:
$$\int |\partial_\phi|^2 + \phi(x) s(x) $$
where the source is two delta functions $s(x) = g\delta(x-x_0) + g\delta(x-x_1)$. Each of these classical sources is steadily spitting out and absorbing particles per unit time at a steady rate g, as you can see by the added source term in the Hamitlonian:
$$ g\phi(x_0) = g\int {d^3k\over 2E_k} e^{ikx_0} \alpha_k + e^{-ikx_0}\alpha^\dagger_k $$
the g term is multiplying a creation operator and an annihilation operator, so the Hamiltonian has a steady amplitude g per unit time to emit any on-shell particle, and the same amplitude to absorb one. If you have no other source, the particles that are absorbed are those emitted by the source, and you just get an (infinite) self-energy renormalization of the mass.
This description is the on-shell old-fasioned perturbation theory, in which the intermediate states are k-states and the description is Hamiltonian in time. This is not covariant, but it shows you that particles are spat out and absorbed, and the two sources only interact to the extent that some of the particles spat out by one are absorbed by the other. The old-fasioned picture is useless for actual computations, but it reveals the particle processes most clearly, because it follows the annihilation and creation of physical particles in detail in time.
The result of the interaction when there are two sources is altered by those particles produced by one, absorbed by the other later. The covariant Schwinger/Feynman form of this introduces particles that meander around in space and time both. Those that do not get absorbed by the other make a field around the particle.
The fact that you are doing things by loop order means that you are not considering the process of a particle emitted by one source absorbed by itself, since this is a loop. The loop order separation of terms makes the scattering process look weird, since it looks like the emitted particle knew where to go to find the other particle. It didn't. If it came back to the first particle, we would include it as part of the next order of Feynman diagram as part of the self-energy graph.