When we are calculating the scattering probability for particles we can do this using Feynman diagrams. For example the tree level diagram for electron positron scattering to two photons is:
(diagram from the question Electron Positron annihilation Feynman Diagram)
The scattering probability is related to the number of nodes in the diagram. For more on this see David Z's answer to Why does electron-positron annihilation prefer to emit photons? In this case the probability is proportional to g2em where gem is the electromagnetic coupling constant.
But suppose we reverse the direction of the time axis to change the diagram to:
Now the diagram shows two photons scattering into an electron-positron pair. But it's the same diagram so the scattering probability is the same i.e. proportional to g2em. The scattering probability is the same in both directions, so to say that:
the electron quantum field "wants" to reduce the energy it has, so when a particle and an anti-particle interact and the charges cancel out (so conservation of charge not violated) the electron field uses the opportunity to get rid of the energy, which is turned into the photon field.
is not a statement that makes much sense.
It is certainly true that in everyday life we frequently see electrons and positron annihilating to photons, but we rarely see two photons annihilating to an electron and positron (NB ignore pair production as this is a different mechanism). However this is not due to any fundamental asymmetry. It's simply because the initial conditions are diferent and in particular photons move at the speed of light while electrons (typically) don't. In fact if we accelerate electrons to speeds near c we find the scattering probability decreases with increasing kinetic energy.