Annihilation can happen when all the quantum numbers of two colliding particles add up to zero. It might be electron on positron, proton on antiproton, neutron on antineutron , quark on antiquark etc.The force responsible depends on the possible interactions of the annihilating particles.
In the case of electron positron annihilation it is primarily the electromagnetic force that is involved and so one gets two photons as an output, usually, in order to conserve quantum numbers and momentum .( a single photon would not conserve spin also as the spin of the electron positron system is even). An annihilation into four photons is very much suppressed by the 1/137 coupling constant entering each photon vertex.
In the case of proton antiproton the main force is the strong force and the products are various hadrons , mesons which conserve quantum numbers, as it is the quarks and antiquarks that disappear and rearange into mesons.
Annihilation does not require the presence of other fields.
Pair production by a single photon needs an external field in order to conserve momentum as @KarsusRen states in his answer. The interaction is electromagnetic.One can think of this as photon photon scattering, where one of the photons is virtual and comes out of the field of the nucleus. Gluons are not free so one cannot observe free creation of antiproton proton pairs, but the diagrams exist.
Here is an interesting measurement of off shell gammma gamma collisions, both photons off shell, generating a proton antiproton pair, which shows how far one can go with the concept of pair production and annihilation,