When one is in the micro level of particles one has to stop thinking classically, i.e. with terms we have developed from our macroscopic observations.
"Touch" at the particle level can be defined as "interaction". Our feeling of touch actually does involve the electromagnetic interactions, we touch with the field of the molecules in our hand the field of the other objects, and this touching involves the exchange of force carrying virtual particles, i.e. particles which do not have definite mass but retain their quantum numbers characterizing them.
There is a precise mathematical description given graphically by the Feynman diagrams:

The diagram translates one to one with integrals that give the probability of this interaction happening. The exchanged electron in this diagram( or positron, depending on how one reads this) is virtual, it does not have definite mass.
Then the closer the e+ and e- are to each other the larger the probability that the scattering will result to an annihilation. So annihilation happens according to the probability of interaction.There are higher order exchanges but they give much smaller probabilities.