It's neither - or everything. Feynman means that the "vertex in the Feynman diagram" which has 2 external lines corresponding to a charged particle and 1 external line corresponding to a photon (all three particles have any energy/frequency and momentum/inverse_wavelength you want) is proportional to $e\approx \sqrt{\alpha}$. So he really means the probability amplitude.
One needs lots of (or at least several) other calculations to calculate the probability. In particular, a stable charged particle can never emit a photon because it would violate the energy or momentum conservation: in its initial rest frame, the energy is minimized, so one can't afford to increase the energy by changing the state of the motion (plus emitting a photon, which would make the final energy even higher). An unstable particle can decay into another particle and a photon. The decay rate will be proportional to the fine-structure constant (without the square root) but the detailed decay rate depends on the mass of the decaying particle as well as the other final decay product.
Also, $\alpha$ slowly depends on a scale - logarithmically. It's the value at $E=0$, or - approximately - anywhere at masses $m<m_e$ where $m_e$ is the electron mass where $\alpha$ has the familiar value $1/137.03604$. Even though I didn't tell you a particular process that is linked to $\alpha$, it's actually a good idea in the process of "renormalization" to define $\alpha$ operationally exactly as a particular probability or amplitude at particular energies, just like you suggested. However, there are many choices how to do that.