As far as I understood, Fermi's golden rule gives a prediction of the transition rate in a perturbed quantum system $H_0+V$ between two eigenstates of the unperturbed system $H_0$, say from $\left| i\right>$ to $\left| f\right>$ with eigenenergies $E_i,E_f$. The prediction is perturbative in the first order of $V$.
The result is, that the rate equals
$$\gamma_{i\to f} = 2\pi |\left<f|V|i\right>|^2 \delta(E_i-E_f)$$
This rate is defined as $$\gamma_{i\to f}=\frac{d}{dt} |\left<f|U(t)|i\right>|^2.$$
My question: How should one interpret this differential "rate" physically?
My thoughts so far: Apparently it is the derivative of the transition probability with respect to time, so it is not a probability itself. In literature it is often called decay rate, but for an exponentially decaying probability $p(t)\sim e^{-\lambda t}$ the decay rate $-\lambda$ would not be computed as $dp/dt$, but $(dp/dt)/p$.
