Gamma radiation is the highest in energy on the electromagnetic spectrum. Any photons with energy over $100keV$ are classified as gamma radiation.
As stated in other answers, for a photon to produce a particle-antiparticle pair, the energy of the photon must be at least twice the mass-energy of the type of particle produced. There are 2 particles of equal mass produced, so it makes sense that you need at least this much energy in the photon that produces them.
The lightest fermion (because we don't usually mean bosons when we talk about pair production) is the electron neutrino; however, neutrinos don't interact with the EM force, so photons won't be directly producing them (indirectly, anything is possible). The lightest non-neutrino fermion is the electron. It's mass-energy is about $0.5MeV$, which means to produce an electron-positron pair, the initial photon has to have an energy of at least about $1MeV$.
Since we already established that anything over $100keV$ is a gamma photon, that means you need a gamma photon to produce an electron-positron pair. Furthermore, since the electron is the lightest particle that photon can produce, any other particle would require the photon to have even more energy. So no non-gamma photon would have enough energy.
This is why we say that pair production only occurs with gamma radiation. Any photon with enough energy to produce a pair of particles must automatically be in the gamma range of the EM spectrum.