Why does pair annihilation occur with particles and only their matching anti-particle? E.g., electrons and positrons, but not protons and positrons? What is the difference?
We (physicists) believe the reason is this: known symmetries and conservation laws. For example, the mutual annihilation of a proton and positron would remove $2\,e$ charge units from the Universe. This violates the conservation of charge principle, which can be seen to arise from the application of Noether's theorem to the global gauge symmetry of the classical electrodynamic Lagrangian, as discussed in QMechanic's answer to the question "Noether theorem and classical proof of electric charge conservation". Other similar conservation laws are conservation of color charges in QCD, conservation of lepton number, baryon number (quark number) and so forth. Your proposed reaction would also violate conservation of lepton number.
But ultimately the answer is simply experimental evidence, i.e. mutual annihilation simply doesn't happen experimentally unless between antiparticles. The failure to observe certain reactions is what led theoretical physicists to define Lagrangians with various continuous symmetries symmetries (and thus, through Noether's theorem, Noether charge (i.e. quantum number) conservations) as well as the notion of anti-particle itself in the first place as a way to organize our knowledge of experimental particle interaction observations. The theoretical reasons above are simply sufficient conditions that would explain our observations, but they are not necessary conditions. It's simply that, so far, our theories agree with experiment and we therefore have no reason to believe they are wrong.